{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "🛠 06. Transfer Learning in TensorFlow Part 3: Scaling-up Exercise Solutions.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "authorship_tag": "ABX9TyO1/gELyZdU+L7WDiiHXHt/",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/ashikshafi08/Learning_Tensorflow/blob/main/Exercise%20Solutions/%F0%9F%9B%A0_06_Transfer_Learning_in_TensorFlow_Part_3_Scaling_up_Exercise_Solutions.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tMNFcPgvp2fV"
      },
      "source": [
        "# 🛠 06. Transfer Learning in TensorFlow Part 3: Scaling-up Exercise Solutions \n",
        "\n",
        "\n",
        "1. Train a feature-extraction transfer learning model for 10 epochs on the same data and compare its performance versus a model which used feature extraction for 5 epochs and fine-tuning for 5 epochs (like we've used in this notebook). Which method is better?\n",
        "\n",
        "2. Recreate the first model (the feature extraction model) with mixed_precision turned on.\n",
        "  - Does it make the model train faster?\n",
        "  - Does it effect the accuracy or performance of our model?\n",
        "  - What's the advatanges of using mixed_precision training?\n",
        "  \n",
        "3. Take 3 of your own photos of food and use the trained model to make predictions on them, share your predictions with the other students in Discord and show off your Food Vision model 🍔👁 . "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "I0a_g64FU660"
      },
      "source": [
        "**Note: ** Using Mixed precision eith EfficientNetB0 model will throw you an error. The error happens when you turn on mixed precision training for    EfficientNetBX models (if you're not sure what this is, you'll see it in the next video).\n",
        "\n",
        "It'll look something like this:\n",
        "\n",
        "```TypeError: Input 'y' of 'Sub' Op has type float16 that does not match type float32 of argument 'x' ```\n",
        "\n",
        "As of 29 May 2021 it looks like this is a bug in TensorFlow 2.5+.\n",
        "\n",
        "There is an issue thread tracking the progress of an update on GitHub: https://github.com/tensorflow/tensorflow/issues/49725\n",
        "\n",
        "**Solution**\n",
        "\n",
        "The current workaround is to **downgrade** to TensorFlow 2.4.1 (the last version before 2.5).\n",
        "\n",
        "You can do this in Google Colab by running:\n",
        "```\n",
        "# Downgrade Tensorflow Version (run this in Google Colab) \n",
        "!pip install tensorflow==2.4.1\n",
        "```\n",
        "After doing so, you'll have to restart your runtime (Runtime -> Restart runtime) to ensure TensorFlow 2.4.1 is active.\n",
        "\n",
        "\n",
        "```\n",
        "# Check if TensorFlow 2.4.1 is active \n",
        "import tensorflow as tf\n",
        "tf.__version__\n",
        "\n",
        ">>> '2.4.1'\n",
        "```\n",
        "\n",
        "Once TensorFlow 2.4.1 is installed, the code should work as normal.\n",
        "\n",
        "If you have any issues, please post your question/reply below so others can see.\n",
        "\n",
        "If a fix for this issue comes for TensorFlow 2.5+ (it should), we'll update this post.\n",
        "\n",
        "Happy mixed precision training,\n",
        "\n",
        "To read the detailed thread: https://github.com/mrdbourke/tensorflow-deep-learning/discussions/82"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "y7z7wJPtWLP6",
        "outputId": "86d2d66e-c2c0-447e-b2c0-a754debf7ef6"
      },
      "source": [
        "# Downgrading tensorflow to 2.4.1 \n",
        "\n",
        "!pip install tensorflow==2.4.1"
      ],
      "execution_count": 1,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Collecting tensorflow==2.4.1\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/70/dc/e8c5e7983866fa4ef3fd619faa35f660b95b01a2ab62b3884f038ccab542/tensorflow-2.4.1-cp37-cp37m-manylinux2010_x86_64.whl (394.3MB)\n",
            "\u001b[K     |████████████████████████████████| 394.3MB 39kB/s \n",
            "\u001b[?25hCollecting tensorflow-estimator<2.5.0,>=2.4.0\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/74/7e/622d9849abf3afb81e482ffc170758742e392ee129ce1540611199a59237/tensorflow_estimator-2.4.0-py2.py3-none-any.whl (462kB)\n",
            "\u001b[K     |████████████████████████████████| 471kB 48.3MB/s \n",
            "\u001b[?25hRequirement already satisfied: absl-py~=0.10 in /usr/local/lib/python3.7/dist-packages (from tensorflow==2.4.1) (0.12.0)\n",
            "Requirement already satisfied: six~=1.15.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow==2.4.1) (1.15.0)\n",
            "Requirement already satisfied: keras-preprocessing~=1.1.2 in /usr/local/lib/python3.7/dist-packages (from tensorflow==2.4.1) (1.1.2)\n",
            "Requirement already satisfied: numpy~=1.19.2 in /usr/local/lib/python3.7/dist-packages (from tensorflow==2.4.1) (1.19.5)\n",
            "Requirement already satisfied: protobuf>=3.9.2 in /usr/local/lib/python3.7/dist-packages (from tensorflow==2.4.1) (3.17.3)\n",
            "Requirement already satisfied: wrapt~=1.12.1 in /usr/local/lib/python3.7/dist-packages (from tensorflow==2.4.1) (1.12.1)\n",
            "Requirement already satisfied: termcolor~=1.1.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow==2.4.1) (1.1.0)\n",
            "Requirement already satisfied: flatbuffers~=1.12.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow==2.4.1) (1.12)\n",
            "Collecting h5py~=2.10.0\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/3f/c0/abde58b837e066bca19a3f7332d9d0493521d7dd6b48248451a9e3fe2214/h5py-2.10.0-cp37-cp37m-manylinux1_x86_64.whl (2.9MB)\n",
            "\u001b[K     |████████████████████████████████| 2.9MB 42.2MB/s \n",
            "\u001b[?25hRequirement already satisfied: wheel~=0.35 in /usr/local/lib/python3.7/dist-packages (from tensorflow==2.4.1) (0.36.2)\n",
            "Requirement already satisfied: astunparse~=1.6.3 in /usr/local/lib/python3.7/dist-packages (from tensorflow==2.4.1) (1.6.3)\n",
            "Collecting grpcio~=1.32.0\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/06/54/1c8be62beafe7fb1548d2968e518ca040556b46b0275399d4f3186c56d79/grpcio-1.32.0-cp37-cp37m-manylinux2014_x86_64.whl (3.8MB)\n",
            "\u001b[K     |████████████████████████████████| 3.8MB 33.9MB/s \n",
            "\u001b[?25hRequirement already satisfied: tensorboard~=2.4 in /usr/local/lib/python3.7/dist-packages (from tensorflow==2.4.1) (2.5.0)\n",
            "Requirement already satisfied: opt-einsum~=3.3.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow==2.4.1) (3.3.0)\n",
            "Collecting gast==0.3.3\n",
            "  Downloading https://files.pythonhosted.org/packages/d6/84/759f5dd23fec8ba71952d97bcc7e2c9d7d63bdc582421f3cd4be845f0c98/gast-0.3.3-py2.py3-none-any.whl\n",
            "Requirement already satisfied: typing-extensions~=3.7.4 in /usr/local/lib/python3.7/dist-packages (from tensorflow==2.4.1) (3.7.4.3)\n",
            "Requirement already satisfied: google-pasta~=0.2 in /usr/local/lib/python3.7/dist-packages (from tensorflow==2.4.1) (0.2.0)\n",
            "Requirement already satisfied: tensorboard-data-server<0.7.0,>=0.6.0 in /usr/local/lib/python3.7/dist-packages (from tensorboard~=2.4->tensorflow==2.4.1) (0.6.1)\n",
            "Requirement already satisfied: google-auth-oauthlib<0.5,>=0.4.1 in /usr/local/lib/python3.7/dist-packages (from tensorboard~=2.4->tensorflow==2.4.1) (0.4.4)\n",
            "Requirement already satisfied: markdown>=2.6.8 in /usr/local/lib/python3.7/dist-packages (from tensorboard~=2.4->tensorflow==2.4.1) (3.3.4)\n",
            "Requirement already satisfied: setuptools>=41.0.0 in /usr/local/lib/python3.7/dist-packages (from tensorboard~=2.4->tensorflow==2.4.1) (57.0.0)\n",
            "Requirement already satisfied: requests<3,>=2.21.0 in /usr/local/lib/python3.7/dist-packages (from tensorboard~=2.4->tensorflow==2.4.1) (2.23.0)\n",
            "Requirement already satisfied: google-auth<2,>=1.6.3 in /usr/local/lib/python3.7/dist-packages (from tensorboard~=2.4->tensorflow==2.4.1) (1.32.1)\n",
            "Requirement already satisfied: werkzeug>=0.11.15 in /usr/local/lib/python3.7/dist-packages (from tensorboard~=2.4->tensorflow==2.4.1) (1.0.1)\n",
            "Requirement already satisfied: tensorboard-plugin-wit>=1.6.0 in /usr/local/lib/python3.7/dist-packages (from tensorboard~=2.4->tensorflow==2.4.1) (1.8.0)\n",
            "Requirement already satisfied: requests-oauthlib>=0.7.0 in /usr/local/lib/python3.7/dist-packages (from google-auth-oauthlib<0.5,>=0.4.1->tensorboard~=2.4->tensorflow==2.4.1) (1.3.0)\n",
            "Requirement already satisfied: importlib-metadata; python_version < \"3.8\" in /usr/local/lib/python3.7/dist-packages (from markdown>=2.6.8->tensorboard~=2.4->tensorflow==2.4.1) (4.6.1)\n",
            "Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests<3,>=2.21.0->tensorboard~=2.4->tensorflow==2.4.1) (2.10)\n",
            "Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests<3,>=2.21.0->tensorboard~=2.4->tensorflow==2.4.1) (3.0.4)\n",
            "Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests<3,>=2.21.0->tensorboard~=2.4->tensorflow==2.4.1) (1.24.3)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests<3,>=2.21.0->tensorboard~=2.4->tensorflow==2.4.1) (2021.5.30)\n",
            "Requirement already satisfied: cachetools<5.0,>=2.0.0 in /usr/local/lib/python3.7/dist-packages (from google-auth<2,>=1.6.3->tensorboard~=2.4->tensorflow==2.4.1) (4.2.2)\n",
            "Requirement already satisfied: rsa<5,>=3.1.4; python_version >= \"3.6\" in /usr/local/lib/python3.7/dist-packages (from google-auth<2,>=1.6.3->tensorboard~=2.4->tensorflow==2.4.1) (4.7.2)\n",
            "Requirement already satisfied: pyasn1-modules>=0.2.1 in /usr/local/lib/python3.7/dist-packages (from google-auth<2,>=1.6.3->tensorboard~=2.4->tensorflow==2.4.1) (0.2.8)\n",
            "Requirement already satisfied: oauthlib>=3.0.0 in /usr/local/lib/python3.7/dist-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<0.5,>=0.4.1->tensorboard~=2.4->tensorflow==2.4.1) (3.1.1)\n",
            "Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.7/dist-packages (from importlib-metadata; python_version < \"3.8\"->markdown>=2.6.8->tensorboard~=2.4->tensorflow==2.4.1) (3.5.0)\n",
            "Requirement already satisfied: pyasn1>=0.1.3 in /usr/local/lib/python3.7/dist-packages (from rsa<5,>=3.1.4; python_version >= \"3.6\"->google-auth<2,>=1.6.3->tensorboard~=2.4->tensorflow==2.4.1) (0.4.8)\n",
            "Installing collected packages: tensorflow-estimator, h5py, grpcio, gast, tensorflow\n",
            "  Found existing installation: tensorflow-estimator 2.5.0\n",
            "    Uninstalling tensorflow-estimator-2.5.0:\n",
            "      Successfully uninstalled tensorflow-estimator-2.5.0\n",
            "  Found existing installation: h5py 3.1.0\n",
            "    Uninstalling h5py-3.1.0:\n",
            "      Successfully uninstalled h5py-3.1.0\n",
            "  Found existing installation: grpcio 1.34.1\n",
            "    Uninstalling grpcio-1.34.1:\n",
            "      Successfully uninstalled grpcio-1.34.1\n",
            "  Found existing installation: gast 0.4.0\n",
            "    Uninstalling gast-0.4.0:\n",
            "      Successfully uninstalled gast-0.4.0\n",
            "  Found existing installation: tensorflow 2.5.0\n",
            "    Uninstalling tensorflow-2.5.0:\n",
            "      Successfully uninstalled tensorflow-2.5.0\n",
            "Successfully installed gast-0.3.3 grpcio-1.32.0 h5py-2.10.0 tensorflow-2.4.1 tensorflow-estimator-2.4.0\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZznCuuG7q43p"
      },
      "source": [
        "## Getting the data and a model readdy"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "320gOoJIvV56",
        "outputId": "52e6432d-7e92-4166-a3d3-6397565782b6"
      },
      "source": [
        "# Getting the helper functions \n",
        "!wget https://raw.githubusercontent.com/mrdbourke/tensorflow-deep-learning/main/extras/helper_functions.py\n",
        "\n",
        "# Import series of helper functions for the notebook (we've created/used these in previous notebooks)\n",
        "from helper_functions import plot_loss_curves, unzip_data, compare_historys, walk_through_dir\n",
        "\n",
        "# Download data from Google Storage (already preformatted)\n",
        "!wget https://storage.googleapis.com/ztm_tf_course/food_vision/101_food_classes_10_percent.zip \n",
        "\n",
        "unzip_data(\"101_food_classes_10_percent.zip\")\n",
        "\n",
        "train_dir = \"101_food_classes_10_percent/train/\"\n",
        "test_dir = \"101_food_classes_10_percent/test/\"\n",
        "\n",
        "# How many images/classes are there?\n",
        "walk_through_dir(\"101_food_classes_10_percent\")"
      ],
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "--2021-07-16 11:35:10--  https://raw.githubusercontent.com/mrdbourke/tensorflow-deep-learning/main/extras/helper_functions.py\n",
            "Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.108.133, 185.199.109.133, 185.199.110.133, ...\n",
            "Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.108.133|:443... connected.\n",
            "HTTP request sent, awaiting response... 200 OK\n",
            "Length: 10246 (10K) [text/plain]\n",
            "Saving to: ‘helper_functions.py’\n",
            "\n",
            "helper_functions.py 100%[===================>]  10.01K  --.-KB/s    in 0s      \n",
            "\n",
            "2021-07-16 11:35:11 (91.6 MB/s) - ‘helper_functions.py’ saved [10246/10246]\n",
            "\n",
            "--2021-07-16 11:35:13--  https://storage.googleapis.com/ztm_tf_course/food_vision/101_food_classes_10_percent.zip\n",
            "Resolving storage.googleapis.com (storage.googleapis.com)... 142.250.188.208, 142.251.33.208, 172.217.13.80, ...\n",
            "Connecting to storage.googleapis.com (storage.googleapis.com)|142.250.188.208|:443... connected.\n",
            "HTTP request sent, awaiting response... 200 OK\n",
            "Length: 1625420029 (1.5G) [application/zip]\n",
            "Saving to: ‘101_food_classes_10_percent.zip’\n",
            "\n",
            "101_food_classes_10 100%[===================>]   1.51G  41.1MB/s    in 35s     \n",
            "\n",
            "2021-07-16 11:35:48 (43.7 MB/s) - ‘101_food_classes_10_percent.zip’ saved [1625420029/1625420029]\n",
            "\n",
            "There are 2 directories and 0 images in '101_food_classes_10_percent'.\n",
            "There are 101 directories and 0 images in '101_food_classes_10_percent/test'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/bibimbap'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/steak'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/breakfast_burrito'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/chicken_wings'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/filet_mignon'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/sashimi'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/garlic_bread'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/escargots'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/bruschetta'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/peking_duck'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/miso_soup'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/frozen_yogurt'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/carrot_cake'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/hot_and_sour_soup'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/lasagna'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/french_onion_soup'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/pulled_pork_sandwich'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/red_velvet_cake'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/clam_chowder'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/grilled_salmon'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/omelette'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/french_fries'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/grilled_cheese_sandwich'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/churros'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/gnocchi'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/spring_rolls'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/ceviche'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/pad_thai'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/edamame'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/beet_salad'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/eggs_benedict'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/bread_pudding'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/onion_rings'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/caprese_salad'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/pork_chop'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/takoyaki'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/strawberry_shortcake'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/ramen'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/beef_tartare'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/shrimp_and_grits'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/caesar_salad'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/baklava'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/spaghetti_bolognese'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/tacos'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/nachos'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/falafel'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/risotto'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/sushi'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/creme_brulee'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/cheese_plate'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/fish_and_chips'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/fried_rice'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/lobster_bisque'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/apple_pie'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/cheesecake'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/paella'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/hummus'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/beignets'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/tuna_tartare'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/mussels'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/panna_cotta'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/ice_cream'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/cannoli'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/hamburger'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/fried_calamari'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/spaghetti_carbonara'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/guacamole'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/hot_dog'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/french_toast'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/pancakes'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/foie_gras'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/pho'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/chocolate_mousse'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/waffles'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/chicken_quesadilla'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/croque_madame'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/chicken_curry'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/lobster_roll_sandwich'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/gyoza'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/prime_rib'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/samosa'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/greek_salad'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/donuts'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/seaweed_salad'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/beef_carpaccio'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/ravioli'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/cup_cakes'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/poutine'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/crab_cakes'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/tiramisu'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/club_sandwich'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/oysters'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/macarons'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/chocolate_cake'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/macaroni_and_cheese'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/pizza'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/scallops'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/dumplings'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/deviled_eggs'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/huevos_rancheros'.\n",
            "There are 0 directories and 250 images in '101_food_classes_10_percent/test/baby_back_ribs'.\n",
            "There are 101 directories and 0 images in '101_food_classes_10_percent/train'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/bibimbap'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/steak'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/breakfast_burrito'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/chicken_wings'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/filet_mignon'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/sashimi'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/garlic_bread'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/escargots'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/bruschetta'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/peking_duck'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/miso_soup'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/frozen_yogurt'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/carrot_cake'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/hot_and_sour_soup'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/lasagna'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/french_onion_soup'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/pulled_pork_sandwich'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/red_velvet_cake'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/clam_chowder'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/grilled_salmon'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/omelette'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/french_fries'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/grilled_cheese_sandwich'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/churros'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/gnocchi'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/spring_rolls'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/ceviche'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/pad_thai'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/edamame'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/beet_salad'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/eggs_benedict'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/bread_pudding'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/onion_rings'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/caprese_salad'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/pork_chop'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/takoyaki'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/strawberry_shortcake'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/ramen'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/beef_tartare'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/shrimp_and_grits'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/caesar_salad'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/baklava'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/spaghetti_bolognese'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/tacos'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/nachos'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/falafel'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/risotto'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/sushi'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/creme_brulee'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/cheese_plate'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/fish_and_chips'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/fried_rice'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/lobster_bisque'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/apple_pie'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/cheesecake'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/paella'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/hummus'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/beignets'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/tuna_tartare'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/mussels'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/panna_cotta'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/ice_cream'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/cannoli'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/hamburger'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/fried_calamari'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/spaghetti_carbonara'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/guacamole'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/hot_dog'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/french_toast'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/pancakes'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/foie_gras'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/pho'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/chocolate_mousse'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/waffles'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/chicken_quesadilla'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/croque_madame'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/chicken_curry'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/lobster_roll_sandwich'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/gyoza'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/prime_rib'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/samosa'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/greek_salad'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/donuts'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/seaweed_salad'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/beef_carpaccio'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/ravioli'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/cup_cakes'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/poutine'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/crab_cakes'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/tiramisu'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/club_sandwich'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/oysters'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/macarons'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/chocolate_cake'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/macaroni_and_cheese'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/pizza'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/scallops'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/dumplings'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/deviled_eggs'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/huevos_rancheros'.\n",
            "There are 0 directories and 75 images in '101_food_classes_10_percent/train/baby_back_ribs'.\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "YJfbOKMnvrh5"
      },
      "source": [
        "# Importing the packages we need \n",
        "import tensorflow as tf\n",
        "from tensorflow.keras import layers \n",
        "from tensorflow.keras.layers.experimental import preprocessing\n",
        "from tensorflow.keras.models import Sequential"
      ],
      "execution_count": 3,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "kmmcHria8rEL"
      },
      "source": [
        "# Create checkpoint callback to save model for later use\n",
        "checkpoint_path = \"101_classes_10_percent_data_model_checkpoint\"\n",
        "checkpoint_callback = tf.keras.callbacks.ModelCheckpoint(checkpoint_path,\n",
        "                                                         save_weights_only=True, \n",
        "                                                         monitor=\"val_accuracy\", \n",
        "                                                         save_best_only=True) "
      ],
      "execution_count": 4,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "CeI_AoOm7qP-",
        "outputId": "dd6bc0fe-5675-452e-e0e8-7c5e5fe4b0c6"
      },
      "source": [
        "# Setup data inputs\n",
        "import tensorflow as tf\n",
        "IMG_SIZE = (224, 224)\n",
        "train_data_all_10_percent = tf.keras.preprocessing.image_dataset_from_directory(train_dir,\n",
        "                                                                                label_mode=\"categorical\",\n",
        "                                                                                image_size=IMG_SIZE)\n",
        "                                                                                \n",
        "test_data = tf.keras.preprocessing.image_dataset_from_directory(test_dir,\n",
        "                                                                label_mode=\"categorical\",\n",
        "                                                                image_size=IMG_SIZE,\n",
        "                                                                shuffle=False) "
      ],
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Found 7575 files belonging to 101 classes.\n",
            "Found 25250 files belonging to 101 classes.\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "6gzyV7WNw8PR",
        "outputId": "354fbb55-9b43-4552-e29b-0ac67225c338"
      },
      "source": [
        "# Building a model to compare with \n",
        "base_model_0 = tf.keras.applications.EfficientNetB0(include_top= False)\n",
        "base_model_0.trainable = False \n",
        "\n",
        "# Setup model architecture with trainable top layers \n",
        "inputs = layers.Input(shape= (224 , 224 , 3) , name='input_layer')\n",
        "x = base_model_0(inputs , training = False)\n",
        "x = layers.GlobalAveragePooling2D(name = 'global_avg_pool_2D')(x)\n",
        "outputs = layers.Dense(len(train_data_all_10_percent.class_names) , activation= 'softmax', name = 'output_layer')(x)\n",
        "\n",
        "# Packing into a model \n",
        "model_0 = tf.keras.Model(inputs , outputs , name = 'model_0')\n",
        "model_0.summary()"
      ],
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Downloading data from https://storage.googleapis.com/keras-applications/efficientnetb0_notop.h5\n",
            "16711680/16705208 [==============================] - 0s 0us/step\n",
            "Model: \"model_0\"\n",
            "_________________________________________________________________\n",
            "Layer (type)                 Output Shape              Param #   \n",
            "=================================================================\n",
            "input_layer (InputLayer)     [(None, 224, 224, 3)]     0         \n",
            "_________________________________________________________________\n",
            "efficientnetb0 (Functional)  (None, None, None, 1280)  4049571   \n",
            "_________________________________________________________________\n",
            "global_avg_pool_2D (GlobalAv (None, 1280)              0         \n",
            "_________________________________________________________________\n",
            "output_layer (Dense)         (None, 101)               129381    \n",
            "=================================================================\n",
            "Total params: 4,178,952\n",
            "Trainable params: 129,381\n",
            "Non-trainable params: 4,049,571\n",
            "_________________________________________________________________\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ECV0HqiN8KyX",
        "outputId": "9c1db473-ff54-4b55-aeb8-9bae5f2dccb4"
      },
      "source": [
        "# Compiling the model\n",
        "model_0.compile(loss=\"categorical_crossentropy\",\n",
        "              optimizer=tf.keras.optimizers.Adam(), \n",
        "              metrics=[\"accuracy\"])\n",
        "\n",
        "# Fit the model \n",
        "history_model_0 = model_0.fit(train_data_all_10_percent,\n",
        "                                           epochs=5, \n",
        "                                           validation_data=test_data,\n",
        "                                           validation_steps=int(0.15 * len(test_data)), \n",
        "                                           callbacks=[checkpoint_callback]) "
      ],
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Epoch 1/5\n",
            "237/237 [==============================] - 85s 202ms/step - loss: 3.7850 - accuracy: 0.1957 - val_loss: 2.4063 - val_accuracy: 0.4740\n",
            "Epoch 2/5\n",
            "237/237 [==============================] - 47s 195ms/step - loss: 1.9215 - accuracy: 0.5932 - val_loss: 2.0266 - val_accuracy: 0.5169\n",
            "Epoch 3/5\n",
            "237/237 [==============================] - 47s 196ms/step - loss: 1.4550 - accuracy: 0.6719 - val_loss: 1.8708 - val_accuracy: 0.5334\n",
            "Epoch 4/5\n",
            "237/237 [==============================] - 47s 196ms/step - loss: 1.2050 - accuracy: 0.7277 - val_loss: 1.7954 - val_accuracy: 0.5418\n",
            "Epoch 5/5\n",
            "237/237 [==============================] - 47s 195ms/step - loss: 1.0208 - accuracy: 0.7761 - val_loss: 1.7618 - val_accuracy: 0.5426\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "9xm4X5RL9ET_",
        "outputId": "fea2298c-4e30-4519-ca9a-098c2775787b"
      },
      "source": [
        "# Evaluating the model \n",
        "results_feature_extraction_model_0 = model_0.evaluate(test_data)\n",
        "results_feature_extraction_model_0"
      ],
      "execution_count": 13,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "790/790 [==============================] - 107s 135ms/step - loss: 1.5922 - accuracy: 0.6063\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[1.592247724533081, 0.606257438659668]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 13
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "bR1LiWIm_XA0"
      },
      "source": [
        "base_model_0.trainable = True \n",
        "\n",
        "# Refreeze every layer except for the last 7 \n",
        "for layer in base_model_0.layers[:-7]:\n",
        "  layer.trainable = False"
      ],
      "execution_count": 14,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "fDuIvPlV_zkI",
        "outputId": "2d7a4b35-8c12-47f9-ca7f-984c5e03622b"
      },
      "source": [
        "# Check which layers are trainable\n",
        "for layer_number, layer in enumerate(base_model_0.layers):\n",
        "  print(layer_number, layer.name, layer.trainable)"
      ],
      "execution_count": 15,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "0 input_1 False\n",
            "1 rescaling False\n",
            "2 normalization False\n",
            "3 stem_conv_pad False\n",
            "4 stem_conv False\n",
            "5 stem_bn False\n",
            "6 stem_activation False\n",
            "7 block1a_dwconv False\n",
            "8 block1a_bn False\n",
            "9 block1a_activation False\n",
            "10 block1a_se_squeeze False\n",
            "11 block1a_se_reshape False\n",
            "12 block1a_se_reduce False\n",
            "13 block1a_se_expand False\n",
            "14 block1a_se_excite False\n",
            "15 block1a_project_conv False\n",
            "16 block1a_project_bn False\n",
            "17 block2a_expand_conv False\n",
            "18 block2a_expand_bn False\n",
            "19 block2a_expand_activation False\n",
            "20 block2a_dwconv_pad False\n",
            "21 block2a_dwconv False\n",
            "22 block2a_bn False\n",
            "23 block2a_activation False\n",
            "24 block2a_se_squeeze False\n",
            "25 block2a_se_reshape False\n",
            "26 block2a_se_reduce False\n",
            "27 block2a_se_expand False\n",
            "28 block2a_se_excite False\n",
            "29 block2a_project_conv False\n",
            "30 block2a_project_bn False\n",
            "31 block2b_expand_conv False\n",
            "32 block2b_expand_bn False\n",
            "33 block2b_expand_activation False\n",
            "34 block2b_dwconv False\n",
            "35 block2b_bn False\n",
            "36 block2b_activation False\n",
            "37 block2b_se_squeeze False\n",
            "38 block2b_se_reshape False\n",
            "39 block2b_se_reduce False\n",
            "40 block2b_se_expand False\n",
            "41 block2b_se_excite False\n",
            "42 block2b_project_conv False\n",
            "43 block2b_project_bn False\n",
            "44 block2b_drop False\n",
            "45 block2b_add False\n",
            "46 block3a_expand_conv False\n",
            "47 block3a_expand_bn False\n",
            "48 block3a_expand_activation False\n",
            "49 block3a_dwconv_pad False\n",
            "50 block3a_dwconv False\n",
            "51 block3a_bn False\n",
            "52 block3a_activation False\n",
            "53 block3a_se_squeeze False\n",
            "54 block3a_se_reshape False\n",
            "55 block3a_se_reduce False\n",
            "56 block3a_se_expand False\n",
            "57 block3a_se_excite False\n",
            "58 block3a_project_conv False\n",
            "59 block3a_project_bn False\n",
            "60 block3b_expand_conv False\n",
            "61 block3b_expand_bn False\n",
            "62 block3b_expand_activation False\n",
            "63 block3b_dwconv False\n",
            "64 block3b_bn False\n",
            "65 block3b_activation False\n",
            "66 block3b_se_squeeze False\n",
            "67 block3b_se_reshape False\n",
            "68 block3b_se_reduce False\n",
            "69 block3b_se_expand False\n",
            "70 block3b_se_excite False\n",
            "71 block3b_project_conv False\n",
            "72 block3b_project_bn False\n",
            "73 block3b_drop False\n",
            "74 block3b_add False\n",
            "75 block4a_expand_conv False\n",
            "76 block4a_expand_bn False\n",
            "77 block4a_expand_activation False\n",
            "78 block4a_dwconv_pad False\n",
            "79 block4a_dwconv False\n",
            "80 block4a_bn False\n",
            "81 block4a_activation False\n",
            "82 block4a_se_squeeze False\n",
            "83 block4a_se_reshape False\n",
            "84 block4a_se_reduce False\n",
            "85 block4a_se_expand False\n",
            "86 block4a_se_excite False\n",
            "87 block4a_project_conv False\n",
            "88 block4a_project_bn False\n",
            "89 block4b_expand_conv False\n",
            "90 block4b_expand_bn False\n",
            "91 block4b_expand_activation False\n",
            "92 block4b_dwconv False\n",
            "93 block4b_bn False\n",
            "94 block4b_activation False\n",
            "95 block4b_se_squeeze False\n",
            "96 block4b_se_reshape False\n",
            "97 block4b_se_reduce False\n",
            "98 block4b_se_expand False\n",
            "99 block4b_se_excite False\n",
            "100 block4b_project_conv False\n",
            "101 block4b_project_bn False\n",
            "102 block4b_drop False\n",
            "103 block4b_add False\n",
            "104 block4c_expand_conv False\n",
            "105 block4c_expand_bn False\n",
            "106 block4c_expand_activation False\n",
            "107 block4c_dwconv False\n",
            "108 block4c_bn False\n",
            "109 block4c_activation False\n",
            "110 block4c_se_squeeze False\n",
            "111 block4c_se_reshape False\n",
            "112 block4c_se_reduce False\n",
            "113 block4c_se_expand False\n",
            "114 block4c_se_excite False\n",
            "115 block4c_project_conv False\n",
            "116 block4c_project_bn False\n",
            "117 block4c_drop False\n",
            "118 block4c_add False\n",
            "119 block5a_expand_conv False\n",
            "120 block5a_expand_bn False\n",
            "121 block5a_expand_activation False\n",
            "122 block5a_dwconv False\n",
            "123 block5a_bn False\n",
            "124 block5a_activation False\n",
            "125 block5a_se_squeeze False\n",
            "126 block5a_se_reshape False\n",
            "127 block5a_se_reduce False\n",
            "128 block5a_se_expand False\n",
            "129 block5a_se_excite False\n",
            "130 block5a_project_conv False\n",
            "131 block5a_project_bn False\n",
            "132 block5b_expand_conv False\n",
            "133 block5b_expand_bn False\n",
            "134 block5b_expand_activation False\n",
            "135 block5b_dwconv False\n",
            "136 block5b_bn False\n",
            "137 block5b_activation False\n",
            "138 block5b_se_squeeze False\n",
            "139 block5b_se_reshape False\n",
            "140 block5b_se_reduce False\n",
            "141 block5b_se_expand False\n",
            "142 block5b_se_excite False\n",
            "143 block5b_project_conv False\n",
            "144 block5b_project_bn False\n",
            "145 block5b_drop False\n",
            "146 block5b_add False\n",
            "147 block5c_expand_conv False\n",
            "148 block5c_expand_bn False\n",
            "149 block5c_expand_activation False\n",
            "150 block5c_dwconv False\n",
            "151 block5c_bn False\n",
            "152 block5c_activation False\n",
            "153 block5c_se_squeeze False\n",
            "154 block5c_se_reshape False\n",
            "155 block5c_se_reduce False\n",
            "156 block5c_se_expand False\n",
            "157 block5c_se_excite False\n",
            "158 block5c_project_conv False\n",
            "159 block5c_project_bn False\n",
            "160 block5c_drop False\n",
            "161 block5c_add False\n",
            "162 block6a_expand_conv False\n",
            "163 block6a_expand_bn False\n",
            "164 block6a_expand_activation False\n",
            "165 block6a_dwconv_pad False\n",
            "166 block6a_dwconv False\n",
            "167 block6a_bn False\n",
            "168 block6a_activation False\n",
            "169 block6a_se_squeeze False\n",
            "170 block6a_se_reshape False\n",
            "171 block6a_se_reduce False\n",
            "172 block6a_se_expand False\n",
            "173 block6a_se_excite False\n",
            "174 block6a_project_conv False\n",
            "175 block6a_project_bn False\n",
            "176 block6b_expand_conv False\n",
            "177 block6b_expand_bn False\n",
            "178 block6b_expand_activation False\n",
            "179 block6b_dwconv False\n",
            "180 block6b_bn False\n",
            "181 block6b_activation False\n",
            "182 block6b_se_squeeze False\n",
            "183 block6b_se_reshape False\n",
            "184 block6b_se_reduce False\n",
            "185 block6b_se_expand False\n",
            "186 block6b_se_excite False\n",
            "187 block6b_project_conv False\n",
            "188 block6b_project_bn False\n",
            "189 block6b_drop False\n",
            "190 block6b_add False\n",
            "191 block6c_expand_conv False\n",
            "192 block6c_expand_bn False\n",
            "193 block6c_expand_activation False\n",
            "194 block6c_dwconv False\n",
            "195 block6c_bn False\n",
            "196 block6c_activation False\n",
            "197 block6c_se_squeeze False\n",
            "198 block6c_se_reshape False\n",
            "199 block6c_se_reduce False\n",
            "200 block6c_se_expand False\n",
            "201 block6c_se_excite False\n",
            "202 block6c_project_conv False\n",
            "203 block6c_project_bn False\n",
            "204 block6c_drop False\n",
            "205 block6c_add False\n",
            "206 block6d_expand_conv False\n",
            "207 block6d_expand_bn False\n",
            "208 block6d_expand_activation False\n",
            "209 block6d_dwconv False\n",
            "210 block6d_bn False\n",
            "211 block6d_activation False\n",
            "212 block6d_se_squeeze False\n",
            "213 block6d_se_reshape False\n",
            "214 block6d_se_reduce False\n",
            "215 block6d_se_expand False\n",
            "216 block6d_se_excite False\n",
            "217 block6d_project_conv False\n",
            "218 block6d_project_bn False\n",
            "219 block6d_drop False\n",
            "220 block6d_add False\n",
            "221 block7a_expand_conv False\n",
            "222 block7a_expand_bn False\n",
            "223 block7a_expand_activation False\n",
            "224 block7a_dwconv False\n",
            "225 block7a_bn False\n",
            "226 block7a_activation False\n",
            "227 block7a_se_squeeze False\n",
            "228 block7a_se_reshape False\n",
            "229 block7a_se_reduce False\n",
            "230 block7a_se_expand True\n",
            "231 block7a_se_excite True\n",
            "232 block7a_project_conv True\n",
            "233 block7a_project_bn True\n",
            "234 top_conv True\n",
            "235 top_bn True\n",
            "236 top_activation True\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "U7-7NpDR_4kq"
      },
      "source": [
        "# Recompiling the model \n",
        "model_0.compile(loss = tf.keras.losses.CategoricalCrossentropy(), \n",
        "                optimizer = tf.keras.optimizers.Adam(learning_rate= 0.001 / 10) , \n",
        "                metrics = ['accuracy'])"
      ],
      "execution_count": 16,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "fqc6hmkMAKrY",
        "outputId": "1f881ca1-1ad7-438e-dc82-0946f7d32d6d"
      },
      "source": [
        "# Fine-tune for 5 more epochs\n",
        "initial_epochs = 5\n",
        "fine_tune_epochs = 5 + initial_epochs # model has already done 5 epochs, this is the total number of epochs we're after (5+5=10)\n",
        "\n",
        "history_model_0_fine_tune = model_0.fit(train_data_all_10_percent,\n",
        "                                                     epochs=fine_tune_epochs,\n",
        "                                                     validation_data=test_data,\n",
        "                                                     validation_steps=int(0.15 * len(test_data)), \n",
        "                                                     initial_epoch=history_model_0.epoch[-1]) "
      ],
      "execution_count": 17,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Epoch 5/10\n",
            "237/237 [==============================] - 54s 206ms/step - loss: 0.2522 - accuracy: 0.9605 - val_loss: 1.9375 - val_accuracy: 0.5482\n",
            "Epoch 6/10\n",
            "237/237 [==============================] - 48s 201ms/step - loss: 0.1941 - accuracy: 0.9721 - val_loss: 1.9648 - val_accuracy: 0.5522\n",
            "Epoch 7/10\n",
            "237/237 [==============================] - 48s 201ms/step - loss: 0.1540 - accuracy: 0.9829 - val_loss: 2.0303 - val_accuracy: 0.5487\n",
            "Epoch 8/10\n",
            "237/237 [==============================] - 48s 200ms/step - loss: 0.1210 - accuracy: 0.9908 - val_loss: 2.0511 - val_accuracy: 0.5516\n",
            "Epoch 9/10\n",
            "237/237 [==============================] - 48s 202ms/step - loss: 0.0941 - accuracy: 0.9948 - val_loss: 2.1201 - val_accuracy: 0.5490\n",
            "Epoch 10/10\n",
            "237/237 [==============================] - 48s 201ms/step - loss: 0.0726 - accuracy: 0.9985 - val_loss: 2.1967 - val_accuracy: 0.5469\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "_YnwoEo0B9_B",
        "outputId": "a54d9bc6-0c35-449e-e553-9dec875b488e"
      },
      "source": [
        "# Evaluating the model results \n",
        "\n",
        "results_fine_tune_model_0 = model_0.evaluate(test_data)\n",
        "results_fine_tune_model_0"
      ],
      "execution_count": 18,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "790/790 [==============================] - 108s 137ms/step - loss: 1.8534 - accuracy: 0.6010\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[1.8533722162246704, 0.6009504795074463]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 18
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "B4mF2cf7Ahix"
      },
      "source": [
        "Alright now we got a model to compare, let's begin our exercise! "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RRl_07jrBKC4"
      },
      "source": [
        "### 1. Train a feature-extraction transfer learning model for 10 epochs on the same data and compare its performance versus a model which used feature extraction for 5 epochs and fine-tuning for 5 epochs (like we've used in this notebook). Which method is better?"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "tTqhTy0rBMb4",
        "outputId": "e61d1d99-07e3-4bcb-b77d-62bc550f5f50"
      },
      "source": [
        "# Building the feature extractor model\n",
        "base_model_1 = tf.keras.applications.EfficientNetB0(include_top= False)\n",
        "base_model_1.trainable = False \n",
        "\n",
        "# Setup model architecture with trainable top layers \n",
        "inputs = layers.Input(shape= (224 , 224 , 3) , name='input_layer')\n",
        "x = base_model_1(inputs , training = False)\n",
        "x = layers.GlobalAveragePooling2D(name = 'global_avg_pool_2D')(x)\n",
        "outputs = layers.Dense(len(train_data_all_10_percent.class_names) , activation= 'softmax', name = 'output_layer')(x)\n",
        "\n",
        "# Packing into a model \n",
        "feature_extractor_model = tf.keras.Model(inputs , outputs , name = 'feature_extractor_model')\n",
        "feature_extractor_model.summary() "
      ],
      "execution_count": 19,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Model: \"feature_extractor_model\"\n",
            "_________________________________________________________________\n",
            "Layer (type)                 Output Shape              Param #   \n",
            "=================================================================\n",
            "input_layer (InputLayer)     [(None, 224, 224, 3)]     0         \n",
            "_________________________________________________________________\n",
            "efficientnetb0 (Functional)  (None, None, None, 1280)  4049571   \n",
            "_________________________________________________________________\n",
            "global_avg_pool_2D (GlobalAv (None, 1280)              0         \n",
            "_________________________________________________________________\n",
            "output_layer (Dense)         (None, 101)               129381    \n",
            "=================================================================\n",
            "Total params: 4,178,952\n",
            "Trainable params: 129,381\n",
            "Non-trainable params: 4,049,571\n",
            "_________________________________________________________________\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "czfU69Y7DGKl",
        "outputId": "7d6b5c66-57e7-41bf-a68e-a6fe7afd74b7"
      },
      "source": [
        "# Compiling the feature extractor model \n",
        "feature_extractor_model.compile(loss = tf.keras.losses.CategoricalCrossentropy() , \n",
        "                                optimizer = tf.keras.optimizers.Adam() , \n",
        "                                metrics = ['accuracy'])\n",
        "\n",
        "# Fitting the feature extractor model for 10 epochs\n",
        "history_feature_extractor_model = feature_extractor_model.fit(train_data_all_10_percent,\n",
        "                                           epochs=10, \n",
        "                                           validation_data=test_data,\n",
        "                                           validation_steps=int(0.15 * len(test_data))) "
      ],
      "execution_count": 20,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Epoch 1/10\n",
            "237/237 [==============================] - 55s 206ms/step - loss: 3.7676 - accuracy: 0.1992 - val_loss: 2.4626 - val_accuracy: 0.4499\n",
            "Epoch 2/10\n",
            "237/237 [==============================] - 48s 201ms/step - loss: 1.9185 - accuracy: 0.5791 - val_loss: 2.0441 - val_accuracy: 0.5154\n",
            "Epoch 3/10\n",
            "237/237 [==============================] - 48s 201ms/step - loss: 1.4653 - accuracy: 0.6679 - val_loss: 1.8590 - val_accuracy: 0.5355\n",
            "Epoch 4/10\n",
            "237/237 [==============================] - 48s 201ms/step - loss: 1.2052 - accuracy: 0.7302 - val_loss: 1.8155 - val_accuracy: 0.5360\n",
            "Epoch 5/10\n",
            "237/237 [==============================] - 48s 201ms/step - loss: 1.0203 - accuracy: 0.7785 - val_loss: 1.7783 - val_accuracy: 0.5400\n",
            "Epoch 6/10\n",
            "237/237 [==============================] - 48s 202ms/step - loss: 0.8897 - accuracy: 0.8190 - val_loss: 1.7334 - val_accuracy: 0.5453\n",
            "Epoch 7/10\n",
            "237/237 [==============================] - 48s 202ms/step - loss: 0.7690 - accuracy: 0.8483 - val_loss: 1.7297 - val_accuracy: 0.5456\n",
            "Epoch 8/10\n",
            "237/237 [==============================] - 48s 201ms/step - loss: 0.6777 - accuracy: 0.8762 - val_loss: 1.7055 - val_accuracy: 0.5543\n",
            "Epoch 9/10\n",
            "237/237 [==============================] - 48s 201ms/step - loss: 0.6009 - accuracy: 0.8970 - val_loss: 1.7156 - val_accuracy: 0.5527\n",
            "Epoch 10/10\n",
            "237/237 [==============================] - 48s 201ms/step - loss: 0.5305 - accuracy: 0.9170 - val_loss: 1.7123 - val_accuracy: 0.5490\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "7want5KfDlz6",
        "outputId": "c3f3d80c-a2e8-4a23-e1b4-ac583f380927"
      },
      "source": [
        "# Evaluating the feature extractor model on test data \n",
        "feature_extractor_results = feature_extractor_model.evaluate(test_data)\n",
        "feature_extractor_results "
      ],
      "execution_count": 21,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "790/790 [==============================] - 109s 137ms/step - loss: 1.4824 - accuracy: 0.6061\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[1.4823750257492065, 0.6061385869979858]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 21
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 573
        },
        "id": "mVKm_EC-EJar",
        "outputId": "dd57ee94-cc59-4f90-b923-54c4fe19aaa1"
      },
      "source": [
        "plot_loss_curves(history_feature_extractor_model)"
      ],
      "execution_count": 22,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEWCAYAAAB2X2wCAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXxU1f3/8ddnJpM9JCF7QkLY10CQyOKCVJQiRbBuiBUVt4rWWuvPr7Zfq61f2/qt1m9rtVIXqlhFLW5gkSKKIhWBEBMg7CBkD0mAbJD9/P6YIQTIBiS5mcnn+XjMIzN37tz5ZJT3nJx77jlijEEppZT7s1ldgFJKqY6hga6UUh5CA10ppTyEBrpSSnkIDXSllPIQGuhKKeUhNNCVUspDaKCrHkFE9ovIZVbXoVRn0kBXSikPoYGueiwR8RGRP4lInuv2JxHxcT0XLiIfi8gRETkkIl+JiM313MMikisi5SKyU0SmWPubKOXkZXUBSlnov4EJQDJggI+AR4FfAQ8COUCEa98JgBGRIcBPgPONMXkikgjYu7ZspZqnLXTVk/0IeMIYc9AYUwT8Bpjreq4WiAH6GmNqjTFfGefER/WADzBcRBzGmP3GmL2WVK/UKTTQVU8WCxxo8viAaxvA08AeYKWI7BORRwCMMXuAnwG/Bg6KyNsiEotS3YAGuurJ8oC+TR4nuLZhjCk3xjxojOkPzAR+fryv3BjzljHmItdrDfC/XVu2Us3TQFc9iUNEfI/fgMXAoyISISLhwGPAPwBEZIaIDBQRAUpxdrU0iMgQEbnUdfK0CjgGNFjz6yh1Mg101ZMsxxnAx2++QCqwGdgCpAFPuvYdBKwCKoB1wF+NMatx9p8/BRQDBUAk8Iuu+xWUapnoAhdKKeUZtIWulFIeQgNdKaU8hAa6Ukp5CA10pZTyEJZd+h8eHm4SExOtenullHJLmzZtKjbGRDT3nGWBnpiYSGpqqlVvr5RSbklEDrT0nHa5KKWUh9BAV0opD6GBrpRSHkLnQ1dKnaS2tpacnByqqqqsLqVH8/X1pU+fPjgcjna/RgNdKXWSnJwcgoKCSExMxDk3mepqxhhKSkrIycmhX79+7X6ddrkopU5SVVVFWFiYhrmFRISwsLAz/itJA10pdRoNc+udzX8Dtwv0PQfLeWLZNmrqdApqpZRqyu0CPfvQMRb+5zvW7CqyuhSllOpW3C7QLxoUTqi/g6UZeVaXopTqBEeOHOGvf/3rGb9u+vTpHDlypNV9HnvsMVatWnW2pTUrMDCwQ493Ltwu0B12G9OTYvh0WyFHa+qsLkcp1cFaCvS6utb/vS9fvpyQkJBW93niiSe47LLLzqm+7swthy3OHB3Lm+uz+HRbIbOS46wuRymP9ZtlmWzLK+vQYw6P7cXjV45o8flHHnmEvXv3kpycjMPhwNfXl9DQUHbs2MGuXbu46qqryM7Opqqqivvvv5+77roLODE/VEVFBVdccQUXXXQRX3/9NXFxcXz00Uf4+flx6623MmPGDK699loSExO55ZZbWLZsGbW1tfzzn/9k6NChFBUVceONN5KXl8fEiRP59NNP2bRpE+Hh4a3+XsYY/uu//otPPvkEEeHRRx9l9uzZ5OfnM3v2bMrKyqirq+PFF1/kggsu4Pbbbyc1NRUR4bbbbuOBBx4458/W7VroAOcn9iYm2Jdl2u2ilMd56qmnGDBgAOnp6Tz99NOkpaXx5z//mV27dgGwcOFCNm3aRGpqKs899xwlJSWnHWP37t3ce++9ZGZmEhISwnvvvdfse4WHh5OWlsb8+fN55plnAPjNb37DpZdeSmZmJtdeey1ZWVntqvv9998nPT2djIwMVq1axUMPPUR+fj5vvfUW3//+9xufS05OJj09ndzcXLZu3cqWLVuYN2/eWX5aJ3PLFrrNJlw5Opa//+c7jhytIcTf2+qSlPJIrbWku8q4ceNOurjmueee44MPPgAgOzub3bt3ExYWdtJr+vXrR3JyMgBjx45l//79zR776quvbtzn/fffB2Dt2rWNx582bRqhoaHtqnPt2rXMmTMHu91OVFQUl1xyCRs3buT888/ntttuo7a2lquuuork5GT69+/Pvn37uO+++/jBD37A1KlT2/+BtMItW+jg7HaprTd8srXA6lKUUp0oICCg8f4XX3zBqlWrWLduHRkZGYwZM6bZi298fHwa79vt9hb734/v19o+52rSpEmsWbOGuLg4br31VhYtWkRoaCgZGRlMnjyZBQsWcMcdd3TIe7ltoI+I7UX/8ACWpmu3i1KeJCgoiPLy8mafKy0tJTQ0FH9/f3bs2ME333zT4e9/4YUX8u677wKwcuVKDh8+3K7XXXzxxbzzzjvU19dTVFTEmjVrGDduHAcOHCAqKoo777yTO+64g7S0NIqLi2loaOCaa67hySefJC0trUNqb7PLRUR8gTWAj2v/JcaYx0/ZxwdYBIwFSoDZxpj9HVJhy3Vx5ehYnvt8N4VlVUT18u3Mt1NKdZGwsDAuvPBCRo4ciZ+fH1FRUY3PTZs2jQULFjBs2DCGDBnChAkTOvz9H3/8cebMmcMbb7zBxIkTiY6OJigoqM3X/fCHP2TdunWMHj0aEeEPf/gD0dHRvP766zz99NM4HA4CAwNZtGgRubm5zJs3j4YG5wWSv//97zukdjHGtL6D8/rTAGNMhYg4gLXA/caYb5rscw8wyhhzt4jcAPzQGDO7teOmpKSYc12xaG9RBVP++CW/mjGc2y9q/wQ2SqmWbd++nWHDhlldhmWqq6ux2+14eXmxbt065s+fT3p6uiW1NPffQkQ2GWNSmtu/zRa6cSZ+heuhw3U79VtgFvBr1/0lwPMiIqatb4tzNCAikJFxvVianquBrpTqEFlZWVx//fU0NDTg7e3Nyy+/bHVJ7dauUS4iYgc2AQOBF4wx60/ZJQ7IBjDG1IlIKRAGFJ9ynLuAuwASEhLOrXKXmaNj+d3yHewvriQxPKDtFyilVCsGDRrEt99+e9K2kpISpkyZctq+n3322WkjbKzUrkA3xtQDySISAnwgIiONMVvP9M2MMS8BL4Gzy+VMX9+cGaOcgb4sI4/7pgzqiEMqpdRJwsLCLOt2ORNnNMrFGHMEWA1MO+WpXCAeQES8gGCcJ0c7XWyIH+P69eajjDw6uYdHKaW6tTYDXUQiXC1zRMQPuBzYccpuS4FbXPevBT7v7P7zpmaOjmXPwQq25zc/1EkppXqC9rTQY4DVIrIZ2Ah8aoz5WESeEJGZrn1eBcJEZA/wc+CRzim3edOTYvCyic7AqJTq0dozymUzMKaZ7Y81uV8FXNexpbVf7wBvLhoUzrKMPB6eNkRXW1FK9Uhue6XoqWYlx5J75BhpWe27qksp5Rlam498//79jBw5sgursZbHBPrlw6Px8bLxkU4FoJTqodxytsXmBPp4cdmwKJZvyeexGcPxsnvMd5VS1vnkESjY0rHHjE6CK55q8elHHnmE+Ph47r33XgB+/etf4+XlxerVqzl8+DC1tbU8+eSTzJo164zetqqqivnz55OamoqXlxfPPvss3/ve98jMzGTevHnU1NTQ0NDAe++9R2xsLNdffz05OTnU19fzq1/9itmzW734vVvwqNS7cnQsxRU1fL23S0ZMKqU6wezZsxsnxwJ49913ueWWW/jggw9IS0tj9erVPPjgg2c8TPmFF15ARNiyZQuLFy/mlltuoaqqigULFnD//feTnp5Oamoqffr0YcWKFcTGxpKRkcHWrVuZNu3Ukdrdk8e00AEmD4kgyNeLpRl5TBocYXU5Srm/VlrSnWXMmDEcPHiQvLw8ioqKCA0NJTo6mgceeIA1a9Zgs9nIzc2lsLCQ6Ojodh937dq13HfffQAMHTqUvn37smvXLiZOnMhvf/tbcnJyuPrqqxk0aBBJSUk8+OCDPPzww8yYMYOLL764s37dDuVRLXRfh51pI6L599YCqmrrrS5HKXWWrrvuOpYsWcI777zD7NmzefPNNykqKmLTpk2kp6cTFRXV7DzoZ+PGG29k6dKl+Pn5MX36dD7//HMGDx5MWloaSUlJPProozzxxBMd8l6dzaMCHWBmcizl1XV8sfOg1aUopc7S7Nmzefvtt1myZAnXXXcdpaWlREZG4nA4WL16NQcOHDjjY1588cW8+eabAOzatYusrCyGDBnCvn376N+/Pz/96U+ZNWsWmzdvJi8vD39/f2666SYeeuihDpuvvLN5VJcLwMT+YYQHerM0I49pI2OsLkcpdRZGjBhBeXk5cXFxxMTE8KMf/Ygrr7ySpKQkUlJSGDp06Bkf85577mH+/PkkJSXh5eXFa6+9ho+PD++++y5vvPEGDoeD6OhofvnLX7Jx40YeeughbDYbDoeDF198sRN+y47X5nzonaUj5kNvya+XZrJ4Qxapj15GkK+jU95DKU/V0+dD707OdD50j+tyAedol+q6BlZmFlpdilJKdRmP63IBOC8hhD6hfizNyOOasX2sLkcp1cm2bNnC3LlzT9rm4+PD+vWnLt3g2Twy0I+vN/rSmn2UVFQTFujT9ouUUo2MMW41J1JSUpJbzFd+Js6mO9wju1zAObdLfYNh+ZZ8q0tRyq34+vpSUlKi6wtYyBhDSUkJvr6+Z/Q6j2yhAwyN7sXgqECWZuQxd2Ki1eUo5Tb69OlDTk4ORUVFVpfSo/n6+tKnz5l1GXtsoINz4YtnVu4i98gx4kL8rC5HKbfgcDjo108XXXdHHtvlAs7RLgAf68IXSqkewKMDvW9YAMnxITqlrlKqR/DoQAdnt8u2/DL2HKywuhSllOpUHh/oM0bFYBN0vVGllMfz+ECP7OXLhP5hLMvI02FYSimP5vGBDs4x6d8VV7Ilt9TqUpRSqtP0iECfNiIGh11YqidHlVIerEcEerC/g0sGR/Lx5nwaGrTbRSnlmXpEoINz4YuCsio27D9kdSlKKdUpekygXzYsEn9vu45JV0p5rB4T6P7eXlw+PIpPtuZTU9dgdTlKKdXhekygg/MioyNHa1m7RycdUkp5nh4V6BcPiiDYz6GjXZRSHqnNQBeReBFZLSLbRCRTRO5vZp/JIlIqIumu22OdU+658fayMT0phpXbCjlWU291OUop1aHa00KvAx40xgwHJgD3isjwZvb7yhiT7Lo90aFVdqCZo2M5WlPPqu263qhSyrO0GejGmHxjTJrrfjmwHYjr7MI6y7h+vYnq5aNzuyilPM4Z9aGLSCIwBmhu5dWJIpIhIp+IyIgOqK1T2G3CjFGxfLmziNKjtVaXo5RSHabdgS4igcB7wM+MMWWnPJ0G9DXGjAb+AnzYwjHuEpFUEUm1cnmrWcmx1NQ3sCJT1xtVSnmOdgW6iDhwhvmbxpj3T33eGFNmjKlw3V8OOEQkvJn9XjLGpBhjUiIiIs6u4uoKWPsnaDj7k5pJccEkhvlrt4tSyqO0Z5SLAK8C240xz7awT7RrP0RknOu4JR1ZaKPtS2HV4/De7VBXc1aHEBFmjo5l3d4SDpZVdXCBSilljfa00C8E5gKXNhmWOF1E7haRu137XAtsFZEM4DngBtNZk48n3whTn4TMD+DtG6Hm6FkdZmZyLA0GPt6s3S5KKc/g1dYOxpi1gLSxz/PA8x1VVJsuuA98esGy++EfV8ON74Bv8BkdYmBkEMNjerE0I4/bLtIVzpVS7s99rxQdewtcuxByUuG1GVBZfMaHmJkcS3r2EbJKzq6Vr5RS3Yn7BjrAyKthzmIo3g1/vwJKc87o5VeOjgVg2WY9OaqUcn/uHegAgy6Hue9DeQEsnAYle9v90rgQP1L6hvJRem4nFqiUUl3D/QMdoO8FcMsyqD3qDPWCre1+6azkWHYVVrCj4NSh9Uop5V48I9ABYpNh3gqwO+C16ZC9oV0vm54Ug92m640qpdyf5wQ6QMRguG0F+IfBolmw9/M2XxIW6MOFA8NZmpFHZ420VEqpruBZgQ4QkgC3/Rt694e3ZsP2ZW2+ZOboWHIOHyMt60gXFKiUUp3D8wIdIDASbv0YYpLh3Zsh/a1Wd//+iCi8vWws06kAlFJuzDMDHcAvFG7+EPpdAh/Oh28WtLhrkK+DKUMj+XhzPnX1ut6oUso9eW6gA3gHOK8iHXYlrHgYvvhfaKGffOboWIorqvlm36EuLlIppTqGZwc6gJcPXPsaJP8Ivvgd/Pu/mw317w2NJNDHS8ekK6XclucHOoDdC2Y+D+PnwzcvwEc/gfq6k3bxddj5/ohoVmQWUF2n640qpdxPzwh0AJsNpv0eJv8C0v8BS+ZBXfVJu8xMjqW8qo4vdlq3+IZSSp2tnhPoACIw+RH4/u+d86ovvgFqKhufvnBAGGEB3rrwhVLKLfWsQD9u4j0w6wXY9wW88UM45hx/7mW3MT0phlXbCqmormv9GEop1c30zEAHGHMTXPca5KY5p9+tOAg453aprmvg020F1tanlFJnqOcGOsDwWc5hjYf2Oif1OpLNeQmhxIX46dwuSim307MDHWDgFJj7oXOBjIXTsB3aw4zRMXy1u5hDlWe3ZqlSSllBAx0gYbxzqoD6alg4jevjDlHXYFi+RdcbVUq5Dw3042JGOaff9fKl/79uYFbvLB3topRyKxroTYUPhNtWIIGRPFP1OH4HVpN35JjVVSmlVLtooJ8qJB7mraCh90BedjxD5qrXra5IKaXaRQO9OYER+Ny+nD2OIVy69ReQ9obVFSmlVJs00FviF8L6i15hbf1IWPoT+Pp5qytSSqlWaaC3Yvp5A7iz7v+xK2wKrPxv+Py3LU6/q5RSVtNAb0VUL1/O6xfJPVX3YsbMhTV/gE8ehgZdBEMp1f1ooLdh5ug49pRUkTn2SZj4E9jwN/jontOm31VKKatpoLfhipHROOzC0s35MPVJ+N6jkLEY/nkL1FZZXZ5SSjXSQG9DaIA3kwZFsCwjjwYDXPIQXPEH2PExLJoFez/XLhilVLfQZqCLSLyIrBaRbSKSKSL3N7OPiMhzIrJHRDaLyHmdU641ZibHkl9axcb9rvVGx/8Yrn4ZSvY4p999PgXWvQDHDltbqFKqR2tPC70OeNAYMxyYANwrIsNP2ecKYJDrdhfwYodWabHLhkXh67CdPBXAqOvh59vg6lcgIAL+/Uv44zD46F7I+9a6YpVSPVabgW6MyTfGpLnulwPbgbhTdpsFLDJO3wAhIhLT4dVaJMDHi8uHR7N8Sz619U26V7x8YNR1cPu/4e61MPoG2PoBvDQZXr4U0t+CWp06QCnVNc6oD11EEoExwPpTnooDsps8zuH00EdE7hKRVBFJLSpyr3U7Z46O5fDRWtbuKW5+h+gkuPJP8OB2uOJpqK6AD+fDs8Ng5a/g0L6uLVgp1eO0O9BFJBB4D/iZMabsbN7MGPOSMSbFGJMSERFxNoewzKTB4fTy9Wp74QvfYBh/F9y7Hm5ZBv0mOfvXnzsP/nEt7FwBDfVdU7RSqkfxas9OIuLAGeZvGmPeb2aXXCC+yeM+rm0ew8fLzhUjY/h4cx7Haurx87a3/gIRZ5j3mwRleZC2CDa9BotnQ3ACpMyDMXMh0L2+2JRS3Vd7RrkI8Cqw3RjzbAu7LQVudo12mQCUGmM8bnWIWcmxVNbU8/mOg2f2wl6xMPkR+NkWuH4R9E6Ez34D/zcc3rsTstbrlAJKqXPWnhb6hcBcYIuIpLu2/RJIADDGLACWA9OBPcBRYF7Hl2q98f3DiAzyYWlGLj8YdRbnfO0O5zqmw2dB0U5IXeg8cbrlXYhKgvNvh6TrwCew44tXSnk8MRa1DFNSUkxqaqol730ufrMskze/yWLjo5cR7Oc49wNWV8DWJbDhFSjcAj69YPQcZ7hHDDn34yulPIqIbDLGpDT3nF4peoZmjo6lpr6Bf2cWdMwBfQJh7K1w91dw20oYPA02/R1eGAevzYDMD6G+tmPeSynl0TTQz1ByfAgJvf1Z1tHrjYo4F6u+5mV4YBtMeRwOH3DOGfN/I2H1750nV5VSqgUa6GdIRJg5Opb/7CmmqLy6c94kMAIu/jncnw5z3nGOcf/yf53B/s5c2PelnkRVSp1GA/0szEyOpcHAvzZ3covZZoch0+CmJfDTNJh4L+z/ChbNdHbJrP8bVJV2bg1KKbehJ0XP0rQ/rcHHYefDey7AObKzi9Qec/arb3wFclPB4Q/9LoFeMRAYDUFRJ/8MiAB7uy43UEq5gdZOiuq/9LN04/gEHvsok4eWbOapq5PwsnfRHzsOP0ie47zlfQsbX4XcTZC9Ho4dOn1/sYF/+OlBHxQNgVHO2/FtDt+u+R2UUp1CA/0szZ3Ql0OVNfxp1W6OHK3hL3POa/vq0Y4WOwZmNVm8uq4GKgqdt/ICqCiA8sKTfxZuhYqDYJqZfsA3uPlW/vHwP/7TJ8h5Elcp1a1ooJ8lEeFnlw0mLMCbx5ZmcvPC9bxy8/kE+3fA2PSz5eUNIfHOW2sa6uFoiSv0Wwj/7G+cP+ubOfHr8IfAyNPDPyASAsKd3Tz+Yc773oEa/kp1EQ30czR3YiKhAd488E46s19ax+u3jSOqVzfvurDZXYEc2fp+xkDVkdNb+U1/FmbC3tVQ3cJ8bV6+zi6fgDBX0Ie7Qj+8yX39AlCqI2igd4AZo2IJ8fPmx2+kcs2LX/PG7ePpFx5gdVnnTgT8Qp23yKGt71tTCZVFUFkCR4td94td94tP3C/a6bxf18I88S19Afi7Hp/0RaBfAEo1paNcOtDmnCPc+veNCPDavHEk9Qm2uqTuq9UvgBLn46b3W/oCsPu4gj7MFfQRzr88gmKcff5BMSe6hbz9u/Z3VKoTtDbKRQO9g+0rqmDuqxs4crSGl29O4YKB4VaX5BlqKk9u6VcWnx76x7dXFEJ9zenH8A12BnxgVJPAj24S/NE62kd1exroXaygtIpbFm7gu+JK/nRDMtOTPGY1PvdgjHPB7vJ85wnf8oIT9ysKmmwrgIZm5snxDTnRsm8a9E2DPyjauQShUl1MA90CpUdruf31jWzKOsz/zBrJTRP6Wl2SOlVDw4ngrzgl+JveKgqbD36/0GZa/DEnRvzYvcFmc14LcDY3m915fqDFfY4/r+cQehK9sMgCwf4O3rh9PPe+lcajH26lpKKGn04Z2LVXlarW2Wyuk69hwMiW92tocF60VZ7vHNnTGPr5rmGf+VC8yxX8dV1W/gkthL7N7rx5BzmvHWjvrbn9bV18jYU6KxroncjP287f5o7l4fc283+rdlFSWc2vrxyBzaah7lZsthOjaqKTWt6vocE1vj8fKg86x/ubhuZvDS1sNw3Oi76MaeW1bTzf9PX1tVBTAdXlzqGlR4vh8H7X43KorWzfZ+AIcIV7YJOg79XkS6CF7T6Bzsd2b+dfOfW1zvMb9TVQX9fkvmt7W/s01J68f4v7tPC8aWjjLyPa/suJtvaR1v+yQmDwVBjxw3P8H/N0GuidzGG38cy1owkP9OGlNfs4VFnDs9cn4+2l86J5HJvNOVOmO60TW1/XJPDLXffLTjxu7XZ0/4l9q8qav/q4o9m8nF8OdoerS8tx4r7d2zlv0fH7Dr8TXyZ2135ia+XL0LWdNp4/7f6ZHMO1T/igTvl4NNC7gM0m/HL6MMICvPn9JzsoPVbLizeNJdBHP35lMbsX+IU4b+fCGKirajn866tPDmK7d5NwbhK4rYa1Q88XtEETpQv9+JIBhAZ484v3t/Cjl79h4a3nExaoIyWUBxBxtogdfm1fgaw6jf7d38WuT4lnwU1j2VFQznV/W0fO4aNWl6SU8hAa6Ba4fHgUb9w+nqLyaq59cR27CsutLkkp5QE00C0yrl9v3v3xRBqM4boF69h0oJm5zJVS6gxooFtoWEwv3pt/AaH+Dn70ynpW7zhodUlKKTemgW6x+N7+LJl/AQMjA7ljUSrvp+VYXZJSyk1poHcD4YE+LL5zAuP79ebn72bwylf7rC5JKeWGNNC7iSBfB3+fdz7Tk6J58l/beeqTHVg1z45Syj3pOPRuxMfLzl/mnEeI/1YWfLmXQ5XV/O6HXbgAtVLKrWmgdzN2m/Dbq0YSHujDc5/t5lBlLc/fOAZfh06OpJRqnTb9uiER4eeXD+Y3M0fw2Y5Cbn51A6XHmpm+VSmlmmgz0EVkoYgcFJGtLTw/WURKRSTddXus48vsmW65IJE/3zCGb7MPM/tv6zhYVmV1SUqpbqw9LfTXgGlt7POVMSbZdXvi3MtSx80cHcvCW88n69BRrlnwNfuL2zndqVKqx2kz0I0xawC9jNFCFw+KYPGdE6ioquPaBV+zNbfU6pKUUt1QR/WhTxSRDBH5RERGtLSTiNwlIqkiklpUVNRBb90zjI4P4Z93X4CPl50bXvqGr/cWW12SUqqb6YhATwP6GmNGA38BPmxpR2PMS8aYFGNMSkSEGy0C0E0MjAxkyfyJxAT7cuvCjazYmm91SUqpbuScA90YU2aMqXDdXw44RCT8nCtTzYoJ9uOfd09kZFwv7nkzjbfWZ1ldklKqmzjnQBeRaHGtfCwi41zHLDnX46qWhfh78+YdE7hkcAS//GALT32yg6M1VixOrJTqTtozbHExsA4YIiI5InK7iNwtIne7drkW2CoiGcBzwA1Gr1nvdH7edl66OYXZKfEs+HIvk5/+grc3ZFFX32B1aUopi4hV2ZuSkmJSU1MteW9Pk7r/EL9bvp20rCMMigzkkSuGcunQSETXX1TK44jIJmNMSnPP6ZWiHiAlsTfvzb+ABTedR12D4fbXU5nz8jdkZB+xujSlVBfSQPcQIsK0kTGsfGAS/zNrBLsLK5j1wn+4b/G3ZJXouqVK9QTa5eKhyqtqeWnNPl7+ah/1DYa5ExK579KBhAZ4W12aUuoctNblooHu4QrLqvi/T3fxbmo2AT5e3DN5IPMuTNTZG5VyU9qH3oNF9fLlqWtGseJnkxiX2Jv/XbGDS5/5gvc25VDfoIORlPIkGug9xOCoIF699XzeunM84UE+PPjPDGb8ZS1rdukUDEp5Cg30HuaCAeF8eM+FPDdnDBXVtdy8cANzX11PZp5O+KWUu9NA74FsNmHm6FhW/fwSfjVjOFtyS5nxl7X8/J10co8cs7o8pdRZ0pOiirqZlPkAABDnSURBVNJjtfz1iz38/T/7AZh3YSL3TB5IsJ/D2sKUUqfRUS6qXXKPHOOPK3fywbe5BPs5+Mn3BjJ3Yl98vHREjFLdhY5yUe0SF+LHs9cn8/F9F5EUF8yT/9rOlD9+yUfpuTToiBiluj0NdHWaEbHBvHH7eBbdNo4gXwf3v53OrBf+o4tqKNXNaaCrFk0aHMHH913EH68bTUlFNTe+vJ7bXtvIzoJyq0tTSjVD+9BVu1TV1vPa1/t5YfUeKqvruG5sPA9cPpjoYF+rS1OqR9GToqrDHK6s4fnVe1i0bj92m3DHRf358SX9CfLVETFKdQUNdNXhskqO8vTKnSzLyKN3gDf3TxnEnHEJeHtpL55SnUkDXXWajOwj/G75dtZ/d4iIIB9mp8Qz+/x44nv7W12aUh5JA111KmMMa3YXs+jr/azeeRADTBoUwY3jE5gyNBIvu7baleooGuiqy+QeOcY7G7N5Z2MWhWXVRPVytdrHJRAX4md1eUq5PQ101eXq6hv4fMdB3tqQxZe7ihBg8pBIbhyXwOQhEdpqV+osaaArS2UfOupstadmU1ReTUywL9enxHPDuHhigrXVrtSZ0EBX3UJtfQOfbS/krQ3ZfLXb2Wq/dGgkN45P4JLBkdhtYnWJSnV7rQW6V1cXo3ouh93GtJExTBsZQ/ahoyzekMW7qTms2p5KXIgfs893jpCJ6qUXKyl1NrSFrixVU9fAqu2FvLU+i7V7irHbhCmuVvvFgyK01a7UKbSFrrotby8b05NimJ4Uw/7iShZvzGJJag4rtxUSF+LHnHHxXJ8ST6S22pVqk7bQVbdTU9fAym0FvLU+i6/3luBlEy4bFsWN4xO4aGA4Nm21qx5MW+jKrXh72ZgxKpYZo2LZV1TB2xuzWbIphxWZBST09ueGcfFcNzaeiCAfq0tVqlvRFrpyC9V19azY6my1r//uEF42YeqIKG4c15cLBoRpq131GDpsUXmUPQcreHtDFkvScjhytJbEMH9uGJfAtWP7EB6orXbl2c4p0EVkITADOGiMGdnM8wL8GZgOHAVuNcaktVWUBro6V1W1J1rtG/YfwmEXvj8imlnJcVw8KBxfh66FqjzPufahvwY8Dyxq4fkrgEGu23jgRddPpTqVr8POVWPiuGpMHLsLy3lrQxbvp+Xy8eZ8/Bx2Jg0OZ+rwaC4dGklogLfV5SrV6drV5SIiicDHLbTQ/wZ8YYxZ7Hq8E5hsjMlv7ZjaQledoaaugfXflfDptkJWZhZSUFaF3SaMS+zN1BFRXD48ij6hOrWvcl/n3IfeRqB/DDxljFnrevwZ8LAx5rS0FpG7gLsAEhISxh44cOAMfg2lzowxhi25pazMLGTltgJ2FVYAMDymF1NHRDF1eDTDYoJw9hoq5R66TaA3pS101dW+K67k020FrMwsZFPWYYyBPqF+TB0ezdQRUaT0DdVZIFW319nj0HOB+CaP+7i2KdWt9AsP4K5JA7hr0gCKyqv5bHshK7cV8o/1B1j4n+8I9XcwZZizW2bSoAj8vPWkqnIvHRHoS4GfiMjbOE+GlrbVf66U1SKCfLhhXAI3jEugsrqONbuKWLmtkJWZBSzZlIOvw8bFgyKYOjyKKcOi6K0nVZUbaDPQRWQxMBkIF5Ec4HHAAWCMWQAsxzlkcQ/OYYvzOqtYpTpDgI8XVyTFcEVSDLX1DWz47hArMwtYua2QT7cVYhM4P7E3U0dEM3V4lK6XqrotvbBIqRYYY9iaW8ZKV7/7zsJyAIbF9OLy4VFMHR7FiNheelJVdSm9UlSpDnCgpLJxOOTGA4cwBuJC/JzhPiKKcYm99aSq6nQa6Ep1sOKKaj7ffpCV2wpYs7uYmroGQvwdXDo0kqnDo5k0OBx/b537TnU8DXSlOlFldR1f7S5iZWYhn+04SOmxWry9bIxNCGXigDAmDghjdJ8QvL209a7OnQa6Ul2ktr6BjfsP8dn2g3y9t4Tt+WUA+DnspCSGMqG/M+BHxQVr94w6KzofulJdxGG3ccGAcC4YEA7A4coa1n9Xwrq9JazbV8LT/94JQIC3nfP79WaiK+BHxAbrcnvqnGmgK9WJQgO8GxfGBmff+zf7TgT8FzuLAAjy9WJ8v96NLfhh0b10jnd1xjTQlepC4YE+jasxARSWVZ0U8Ku2HwQgxN/B+MYWfDiDowJ1eKRqk/ahK9WN5B051hju6/aWkHvkGABhAd5M6B/GhAFhTOwfxoCIAA34HkpPiirlprIPHT0p4AvKqgCIDPJp7J6Z2D+MvmH+GvA9hJ4UVcpNxff2J763P9efH48xhv0lJwL+670lLM3IAyAm2JeJTVrwOj1Bz6QtdKXclDGGvUUVrNtbwjf7DvHNvhJKKmsA57TAE/uHMaF/GGMSQugXrl00nkK7XJTqARoaDLsPVrBubzHr9jlDvvRYLQDBfg5G9QlmTHwIyQkhjO4TQpguqO2WNNCV6oEaGgy7DpaTkX2E9OwjpGeXsrOgjAbXP/n43n4kx4cyuk8wYxJCGBEbrAtruwENdKUUAEdr6tiSU0pGjivks46QV+o80eplE4bGBJEc72zBj0kIoX94oI6H72Y00JVSLTpYVkV69pHGkN+cXUp5dR0AQT5ejIoPbgz55IQQIoN8La64Z9NAV0q1W0ODYV9xBd9mnQj5Hfnl1Ln6auJC/BjtCvnk+FBGxvXSmSW7kA5bVEq1m80mDIwMYmBkENelOJcLrqqtJzOv1BXypaRnH2b5lgIA7DZhcFQQyU1CfmBkoM5NYwENdKVUm3wddsb27c3Yvr0btxVXVLM5x9kPn55TyvItBSzekA04Jx9L6hNMcnwoyfHBjOoTQkywrw6d7GQa6EqpsxIe6MOlQ6O4dGgU4BwX/11xpbObxhXyr67dR229s6sm2M/B0OgghsX0YlhMEEOjezE4Kgg/bx1Z01E00JVSHUJE6B8RSP+IQH44pg8A1XX1bMsrY2teGTvyy9ieX8Y/U7OprKkHwCaQGB7AsOhejWE/NCaIuBA/bc2fBQ10pVSn8fGyMyYhlDEJoY3bGhoMOYePsS2/jB0FZezIL2drXin/2pLfuE+Qr5cz5F0t+WExQQyJDtKTr23QT0cp1aVsNiEhzJ+EMH+mjYxu3F5ZXcfOwnK25ztDfkdBGR+k5VJefQAAEejb29/ZineF/fCYXsSF+OlYeRcNdKVUtxDg48V5CaGc16Q1b4yzNb+jwBX0rhb9iswCjo+4DvTxYkh0UGO/vLM134tAn54XbzoOXSnldo7W1LGrsKKxX357QTk78ssoq6pr3Ceht/9pJ2ETevu7fWtex6ErpTyKv7eXa8x7SOM2Ywx5pVXsyC9jR0G5s48+v4xV2wsb56/xddhIDAtgQEQgAyIC6B8RyICIQPpFBHhEi979fwOllMI5yiYuxI+4ED+mDItq3F5VW88uV9/8noMV7CuqJDOvlE+25jcGPUB0L1/6RwTQP8IZ+P1doR8b7D599BroSimP5uuwM6pPCKP6hJy0vbqunqySo+wtqmRvkTPo9xZVsDQ976SuG1+HjX7hgc6gDw9gQGQg/V2PA7pZq757VaOUUl3Ex8vOoKggBkUFnbTdGENJZQ17D1awr7iy8efW3FI+2dJ8q97Zoj/x06pWvQa6Uko1ISKEB/oQHujD+P5hJz1XXVfPgZKj7CuqOKll/2F6LuUttepdXTcDIgLpF965rfp2HVlEpgF/BuzAK8aYp055/lbgaSDXtel5Y8wrHVinUkpZzsfLzuCoIAY306ovrqg5qetmX1EFW3Kab9XfflE/7pzUv8PrazPQRcQOvABcDuQAG0VkqTFm2ym7vmOM+UmHV6iUUt2ciBAR5ENEkA8TTmnVV9WeaNUf78KJ7NU5y/+1p4U+DthjjNkHICJvA7OAUwNdKaXUKXwddoZEO6cu6Gy2duwTB2Q3eZzj2naqa0Rks4gsEZH45g4kIneJSKqIpBYVFZ1FuUoppVrSnkBvj2VAojFmFPAp8HpzOxljXjLGpBhjUiIiIjrorZVSSkH7Aj0XaNri7sOJk58AGGNKjDHVroevAGM7pjyllFLt1Z5A3wgMEpF+IuIN3AAsbbqDiMQ0eTgT2N5xJSqllGqPNk+KGmPqROQnwL9xDltcaIzJFJEngFRjzFLgpyIyE6gDDgG3dmLNSimlmqGzLSqllBtpbbbFjjopqpRSymIa6Eop5SEs63IRkSLgwFm+PBwo7sBy3J1+HifTz+ME/SxO5gmfR19jTLPjvi0L9HMhIqkt9SH1RPp5nEw/jxP0sziZp38e2uWilFIeQgNdKaU8hLsG+ktWF9DN6OdxMv08TtDP4mQe/Xm4ZR+6Ukqp07lrC10ppdQpNNCVUspDuF2gi8g0EdkpIntE5BGr67GSiMSLyGoR2SYimSJyv9U1WU1E7CLyrYh8bHUtVhORENf6BDtEZLuITLS6JquIyAOufyNbRWSxiPhaXVNncKtAb7Ic3hXAcGCOiAy3tipL1QEPGmOGAxOAe3v45wFwPzrb53F/BlYYY4YCo+mhn4uIxAE/BVKMMSNxTjJ4g7VVdQ63CnSaLIdnjKkBji+H1yMZY/KNMWmu++U4/8E2t5pUjyAifYAf4JyTv0cTkWBgEvAqgDGmxhhzxNqqLOUF+ImIF+AP5FlcT6dwt0Bv73J4PY6IJAJjgPXWVmKpPwH/BTRYXUg30A8oAv7u6oJ6RUQCrC7KCsaYXOAZIAvIB0qNMSutrapzuFugq2aISCDwHvAzY0yZ1fVYQURmAAeNMZusrqWb8ALOA140xowBKoEeec5JREJx/iXfD4gFAkTkJmur6hzuFuhtLofX04iIA2eYv2mMed/qeix0ITBTRPbj7Iq7VET+YW1JlsoBcowxx/9iW4Iz4Huiy4DvjDFFxpha4H3gAotr6hTuFuhtLofXk4iI4Owj3W6MedbqeqxkjPmFMaaPMSYR5/8XnxtjPLIV1h7GmAIgW0SGuDZNAbZZWJKVsoAJIuLv+jczBQ89QdzmEnTdSUvL4VlclpUuBOYCW0Qk3bXtl8aY5RbWpLqP+4A3XY2ffcA8i+uxhDFmvYgsAdJwjgz7Fg+dAkAv/VdKKQ/hbl0uSimlWqCBrpRSHkIDXSmlPIQGulJKeQgNdKWU8hAa6MrjiEi9iKQ3uXXYFZIikigiWzvqeEp1JLcah65UOx0zxiRbXYRSXU1b6KrHEJH9IvIHEdkiIhtEZKBre6KIfC4im0XkMxFJcG2PEpEPRCTDdTt+ubhdRF52za+9UkT8XPv/1DU3/WYReduiX1P1YBroyhP5ndLlMrvJc6XGmCTgeZyzMwL8BXjdGDMKeBN4zrX9OeBLY8xonPOgHL8qeRDwgjFmBHAEuMa1/RFgjOs4d3fWL6dUS/RKUeVxRKTCGBPYzPb9wKXGmH2uSc0KjDFhIlIMxBhjal3b840x4SJSBPQxxlQ3OUYi8KkxZpDr8cOAwxjzpIisACqAD4EPjTEVnfyrKnUSbaGrnsa0cP9MVDe5X8+Jc1E/wLmi1nnARtdiCkp1GQ101dPMbvJznev+15xYkuxHwFeu+58B86FxrdLglg4qIjYg3hizGngYCAZO+ytBqc6kLQjlifyazD4JznU1jw9dDBWRzThb2XNc2+7DubLPQzhX+Tk+K+H9wEsicjvOlvh8nCveNMcO/MMV+gI818OXfFMW0D501WO4+tBTjDHFVteiVGfQLhellPIQ2kJXSikPoS10pZTyEBroSinlITTQlVLKQ2igK6WUh9BAV0opD/H/AZs2uY47L0SDAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEWCAYAAAB2X2wCAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXxU1f3/8dcnkx1CdgJZIAiRfY8sRVFBLa64FIGiX1eoda22VWtbpZa2/qzWaotWXEBxQVywaHFXRAUkAdlBCGsmYckeAllnzu+PGeIQEzJAkjsz+Twfjzxy5947dz4Zkjdnzj33XDHGoJRSyv8FWV2AUkqplqGBrpRSAUIDXSmlAoQGulJKBQgNdKWUChAa6EopFSA00JVSKkBooCu/IyJLRaRERMKsrkUpX6KBrvyKiKQDZwEGuKwNXze4rV5LqZOlga78zf8BK4F5wHVHV4pImoi8IyIFIlIkIv/22DZdRLaIyCER2Swiw9zrjYj08thvnojMci+fIyJ2EblPRPYDc0UkVkTed79GiXs51eP5cSIyV0Ty3dvfda/fKCKXeuwXIiKFIjK01d4l1S5poCt/83/Aq+6vn4pIkojYgPeBPUA6kAIsABCRScBM9/M64WrVF3n5Wl2AOKA7MAPX38tc9+NuQCXwb4/95wORQH+gM/CEe/3LwDUe+10E7DPGfOdlHUp5RXQuF+UvRORM4AugqzGmUES2As/iarEvdq+va/Ccj4AlxpgnGzmeATKMMTnux/MAuzHmDyJyDvAx0MkYU9VEPUOAL4wxsSLSFcgD4o0xJQ32Swa+B1KMMeUi8hawyhjz6Em/GUo1Qlvoyp9cB3xsjCl0P37NvS4N2NMwzN3SgB0n+XoFnmEuIpEi8qyI7BGRcmAZEOP+hJAGFDcMcwBjTD7wDXCViMQAF+L6hKFUi9ITPcoviEgEcDVgc/dpA4QBMcABoJuIBDcS6rlAzyYOewRXF8lRXQC7x+OGH19/DfQGRhpj9rtb6N8B4n6dOBGJMcaUNvJaLwE34/qbW2GMyWv6p1Xq5GgLXfmLywEH0A8Y4v7qC3zl3rYPeEREOohIuIiMcT/veeA3IjJcXHqJSHf3trXAz0XEJiITgLObqSEKV795qYjEAQ8d3WCM2Qd8ADztPnkaIiJjPZ77LjAMuAtXn7pSLU4DXfmL64C5xpi9xpj9R79wnZScClwK9AL24mplTwYwxrwJ/AVX98whXMEa5z7mXe7nlQLT3NuO559ABFCIq9/+wwbbrwVqga3AQeBXRzcYYyqBt4EewDsn+LMr5RU9KapUGxGRB4HTjTHXNLuzUidB+9CVagPuLpqbcLXilWoV2uWiVCsTkem4Tpp+YIxZZnU9KnBpl4tSSgUIbaErpVSAsKwPPSEhwaSnp1v18kop5ZdWr15daIxJbGybZYGenp5Odna2VS+vlFJ+SUT2NLVNu1yUUipAaKArpVSA0EBXSqkA4VMXFtXW1mK326mqanS2UhVgwsPDSU1NJSQkxOpSlAoIPhXodrudqKgo0tPTERGry1GtyBhDUVERdrudHj16WF2OUgHBp7pcqqqqiI+P1zBvB0SE+Ph4/TSmVAvyqUAHNMzbEf23Vqpl+VSXi1JKBaqCQ9VszCtjvb2M8X07MyAlusVfQwNdKaVaWPHhGjbklbkDvJQN9jLyy1zdiyIQ3zFUA721lZaW8tprr3Hrrbee0PMuuugiXnvtNWJiYprc58EHH2Ts2LGcd955p1qmUsqHlB2pZWO+q+W9Ia+U9fYy7CWV9dtPS+jAGT3iGJgSzaDUGPold6JjWOtEr2WzLWZmZpqGl/5v2bKFvn37WlIPwO7du7nkkkvYuHHjMevr6uoIDm6///e15s9v9b+5UifiUFUtm/LL2WAvY31eGRvspewuOlK/vVtcJANToxmUEs3A1GgGpETTKbxlh+WKyGpjTGZj27z6K3Xfb/FJwAY8b4x5pMH27sCLQCJQDFxjjLH/6EAn4E/vbWJzfvmpHOJH+iV34qFL+ze5/f7772fHjh0MGTKEkJAQwsPDiY2NZevWrWzbto3LL7+c3NxcqqqquOuuu5gxYwbww7w0FRUVXHjhhZx55pksX76clJQU/vvf/xIREcH111/PJZdcws9+9jPS09O57rrreO+996itreXNN9+kT58+FBQU8POf/5z8/HxGjx7NJ598wurVq0lISGi03qbq+fDDD3nggQdwOBwkJCTw2WefUVFRwR133EF2djYiwkMPPcRVV11Fx44dqaioAOCtt97i/fffZ968eVx//fWEh4fz3XffMWbMGKZMmcJdd91FVVUVERERzJ07l969e+NwOLjvvvv48MMPCQoKYvr06fTv35+nnnqKd9913dHtk08+4emnn2bRokUt+c+pVKs6UlPHpvxyV8vbXsr6vDJ2FR7maBs4JSaCgSnRTMpMY1BqNANToomJDLW05mYDXURswGzgfFz3aswSkcXGmM0euz0GvGyMeUlExgF/ww/vzPLII4+wceNG1q5dy9KlS7n44ovZuHFj/TjpF198kbi4OCorKznjjDO46qqriI+PP+YY27dv5/XXX+e5557j6quv5u233+aaa358x7GEhATWrFnD008/zWOPPcbzzz/Pn/70J8aNG8fvfvc7PvzwQ1544YXj1ttYPU6nk+nTp7Ns2TJ69OhBcXExAH/+85+Jjo5mw4YNAJSUlDT7ftjtdpYvX47NZqO8vJyvvvqK4OBgPv30Ux544AHefvtt5syZw+7du1m7di3BwcEUFxcTGxvLrbfeSkFBAYmJicydO5cbb7zRq38DpaxQVetg8z53y9vddZJzsAKnO7yTOoUxMCWGy4ekMNAd3gkdw6wtuhHetNBHADnGmJ0AIrIAmAh4Bno/4B738hc0f7PdZh2vJd1WRowYccxFL0899VR9KzM3N5ft27f/KNB79OjBkCFDABg+fDi7d+9u9NhXXnll/T7vvOO6Z/DXX39df/wJEyYQGxt73Poaq6egoICxY8fW1x0X57of8qeffsqCBQvqn9vcsQEmTZqEzWYDoKysjOuuu47t27cjItTW1tYf95Zbbqnvkjn6etdeey2vvPIKN9xwAytWrODll/VG98o3VNc52LrvUH2XyYa8crYdOITDnd4JHUMZlBrDhQO61re8O3cKt7hq73gT6Cm4bp91lB0Y2WCfdcCVuLplrgCiRCTeGFPkuZOIzABmAHTr1u1ka24zHTp0qF9eunQpn376KStWrCAyMpJzzjmn0YtiwsJ++F/bZrNRWVn5o30897PZbNTV1Z1wbd7W0xzPseANn+/58//xj3/k3HPPZdGiRezevZtzzjnnuMe94YYbuPTSSwkPD2fSpEnt+hyEsk6dw8n2gxWsyy1lnbvl/f3+Q9Q6XOEdGxnCwNQYxvfp7Or7To2mS6dwv71GoqX+yn4D/FtErgeWAXmAo+FOxpg5wBxwnRRtodduMVFRURw6dKjRbWVlZcTGxhIZGcnWrVtZuXJli7/+mDFjWLhwIffddx8ff/zxcbtFmqpn1KhR3Hrrrezatau+yyUuLo7zzz+f2bNn889//hNwdbnExsaSlJTEli1b6N27N4sWLSIqKqrJ10tJSQFg3rx59evPP/98nn32Wc4999z6Lpe4uDiSk5NJTk5m1qxZfPrppy30DinVNGMMe4qOsM5eyrpc13DBjfllVNU6AYgKD2ZQajQ3nXlafcs7NTbCb8O7Md4Eeh6Q5vE41b2unjEmH1cLHRHpCFxljCltqSLbSnx8PGPGjGHAgAFERESQlJRUv23ChAn85z//oW/fvvTu3ZtRo0a1+Os/9NBDTJ06lfnz5zN69Gi6dOnSZMA2VU9iYiJz5szhyiuvxOl00rlzZz755BP+8Ic/cNtttzFgwABsNhsPPfQQV155JY888giXXHIJiYmJZGZm1p8gbejee+/luuuuY9asWVx88cX162+++Wa2bdvGoEGDCAkJYfr06dx+++0ATJs2jYKCAh3FolrFgfIqd8vbNVRwvb2MskpXV2BYcBD9kzsxdUQ3BqfGMCg1mvT4DgQFBU54N6bZYYsiEgxsA8bjCvIs4OfGmE0e+yQAxcYYp4j8BXAYYx483nF9cdii1aqrq7HZbAQHB7NixQp++ctfsnbtWqvLOmm33347Q4cO5aabbmpyn/b+b668U3aklvXuMd5HQ/xAeTUAtiDh9KQoBqdGMzjNFd6nJ0URYvO5mU1axCkNWzTG1InI7cBHuIYtvmiM2SQiDwPZxpjFwDnA30TE4Opyua3Fqm9H9u7dy9VXX43T6SQ0NJTnnnvO6pJO2vDhw+nQoQOPP/641aUoP1NZ42BTfhnr7K5uk/V213DBo3okdGDUafEMSo1hSFo0/bpGExFqs7Bi36EXFvm4oqIixo8f/6P1n3322Y9G2Pgj/Tdv32odTr7ff8jdZeI6cek54qRLp3AGebS8B6XEEB3ZvufPP+ULi5R14uPj/brbRamjnE7D7qLDx5y03JRfTnWd66RldEQIg1KjGd+nZ32IJ/nJcEFfoYGulGoV1XUOvttbyvIdRazeU8x6exmHqlxDdCNCbAxI6cQ1o7ozKDWaIWkxdIuLDKgRJ1bQQFdKtQiH07Apv4xvcopYvqOQrN3FVNU6CRLo27UTlw5OZnCqa4KqjM4dCQ7Qk5ZW0kBXSp0UYww7CipYvqOIb3IKWbmzuH7Y4OlJHZlyRjd+0jOekafFEx3Rvvu924oGulLKa/mllXyTU8jyHa5W+NGhgykxEfy0fxJjeiUwumc8naO079sKGuinwHOmQqUCUfHhGla4w3v5jqL64YPxHUIZ3TOeMb0SGNMzgbS4wLri0l9poAeA9j5fu2o5h6vrWLW7mOU5hXyTU8Tmfa4prDuGBTOyRxzTRnZjTK8EeidFBfxVl/7Id1Pgg/th/4aWPWaXgXDhI01uvv/++0lLS+O221zXRc2cOZPg4GC++OILSkpKqK2tZdasWUycOLHZl6qoqGDixImNPu/ll1/mscceQ0QYNGgQ8+fP58CBA9xyyy3s3LkTgGeeeYbk5ORjbrjx2GOPUVFRwcyZMznnnHMYMmQIX3/9NVOnTuX0009n1qxZ1NTUEB8fz6uvvkpSUlKj86CXlZWxfv36+nldnnvuOTZv3swTTzxxSm+v8j81dU6+21vCNzuKWJ5TyNrcUuqchlBbEMO6x/Dr80/nJ70SGJQaHbBXXgYS3w10C0yePJlf/epX9YG+cOFCPvroI+688046depEYWEho0aN4rLLLmv242V4eDiLFi360fM2b97MrFmzWL58OQkJCfXzld95552cffbZLFq0CIfDQUVFRbNzltfU1HD04qySkhJWrlyJiPD888/z6KOP8vjjjzc6D3pISAh/+ctf+Pvf/05ISAhz587l2WefPdW3T/kBh9OwOb+cb3YU8k1OIdm7S6isdRAkMDAlmuljT2NMzwQy02MJD9GrL/2N7wb6cVrSrWXo0KEcPHiQ/Px8CgoKiI2NpUuXLtx9990sW7aMoKAg8vLyOHDgAF26dDnusYwxPPDAAz963ueff86kSZPq70J0dP7wzz//vH7OcJvNRnR0dLOBPnny5Pplu93O5MmT2bdvHzU1NfXzoTc1D/q4ceN4//336du3L7W1tQwcOPAE3y3lD1wjUQ67+sBzilixs6h+JEpG545MPiNNR6IEEN8NdItMmjSJt956i/379zN58mReffVVCgoKWL16NSEhIaSnp3s17/jJPs9TcHAwTqez/vHx5iu/4447uOeee7jssstYunQpM2fOPO6xb775Zv7617/Sp08fbrjhhhOqS/m2oopqvs4pZNk2Vyt8f7nr9yYlJoIL+rlGovykZ7zf3LRBeU8DvYHJkyczffp0CgsL+fLLL1m4cCGdO3cmJCSEL774gj179nh1nLKyskafN27cOK644gruuece4uPj6+cPHz9+PM888wy/+tWv6rtckpKSOHjwIEVFRXTs2JH333+fCRMmNPl6R+crf+mll+rXNzUP+siRI8nNzWXNmjWsX7/+VN4yZbGaOier95Tw1fYClm0vYGOe60RmTGRI/SiUMb3i9UrMdkADvYH+/ftz6NAhUlJS6Nq1K9OmTePSSy9l4MCBZGZm0qdPH6+O09Tz+vfvz+9//3vOPvtsbDYbQ4cOZd68eTz55JPMmDGDF154AZvNxjPPPMPo0aN58MEHGTFiBCkpKcd97ZkzZzJp0iRiY2MZN24cu3btAmhyHnSAq6++mrVr13p1OzrlO4wx7Co8zFfbC1m2rYAVO4s4UuMgOEgY1i2WX59/OmNPT2RASjQ2HYnSruhsi+3YJZdcwt13393obI5tRf/NvVNWWcvynEKWbS/kq+0F2EtctzbsHh/J2IxEzspwXdATFa794IFOZ1tUxygtLWXEiBEMHjzY0jBXTatzOFlnL3N1o2wrYG1uKU7jGg8+umc8vzi7J2MzEuge36H5g6l2QwP9FG3YsIFrr732mHVhYWF8++23FlXUvJiYGLZt22Z1GaoBe8mR+m6Ub3IKKa+qQwQGpURz27m9OCsjkaHdYnQ8uGqSzwW6McavTtwMHDhQ5ys/SVZ19/mKw9V1rNxZVB/iO92X1XfpFM6EAV0Ye3oiY3omENsh1OJKlb/wqUAPDw+nqKiI+Ph4vwp1deKMMRQVFREe3n6Gzjmdhs37ylnm7kZZvaeEWochPCSIkT3imTaqO2MzEujVuaP+/quT4lOBnpqait1up6CgwOpSVBsIDw8nNTXV6jJa1cHyqvoTmV9vL6TocA0AfbpEceOYHpyVkahXZaoW41OBHhISUn+Fo1L+asu+chZ9l8eybQVs3X8IcM1OeFZGAme5R6ToRT2qNfhUoCvlr6rrHHywYT/zV+5h9Z4SQmxCZvc47p3Qm7EZifTr2klnJ1StTgNdqVOQW3yEV7/dy8LsXIoP15AeH8nvL+rLz4an6slM1eY00JU6QQ6n4cttB5m/Yg9LtxUgwHl9k7hmVHfO7JWgLXFlGQ10pbxUWFHNwuxcXl25l7zSShKjwrjj3F5MGdGN5JgIq8tTyrtAF5EJwJOADXjeGPNIg+3dgJeAGPc+9xtjlrRwrUq1OWMM2XtKeGXlHpZs2EetwzDqtDgeuKgvF/RP0ot8lE9pNtBFxAbMBs4H7ECWiCw2xmz22O0PwEJjzDMi0g9YAqS3Qr1KtYmK6joWfZfHqyv3sHX/IaLCgpk2sjvTRnYjIynK6vKUapQ3LfQRQI4xZieAiCwAJgKegW6ATu7laCC/JYtUqq1s3V/OKyv3sGhNHodrHPRP7sQjVw7ksiHJRIZqD6Xybd78hqYAuR6P7cDIBvvMBD4WkTuADsB5jR1IRGYAMwC6det2orUq1Sqq6xx8uHE/r6zcQ9buEkKDg7hkUFeuHdWdIWkxetWm8hst1eSYCswzxjwuIqOB+SIywBjj9NzJGDMHmAOu6XNb6LWVOim5xUd4fdVe3sjKpehwDd3iInngoj5MGp6mQw6VX/Im0POANI/Hqe51nm4CJgAYY1aISDiQABxsiSKVaikOp2HZtgJeWbmHz78/iADj3UMOz9Ihh8rPeRPoWUCGiPTAFeRTgJ832GcvMB6YJyJ9gXBAJ2RRPqOoopqF2XZeW7WH3OJKEjqGcbt7yGGKDjlUAaLZQDfG1InI7cBHuIYkvmiM2SQiDwPZxpjFwK+B50TkblwnSK837X1uVGU5Ywxr9pYwf8UelmzYT43Dycgecdw3oQ8X9OtCaLAOOVSBxas+dPeY8iUN1j3osbwZGNOypSl1cg5X1/Hu2jzmr/hhyOHUEWlMG9Wd03XIoQpgOg5LBYy9RUd44eudvL0mj4rqOvp17cRfrxjIxCHJdAjTX3UV+PS3XPm97/cf4pmlOby3fh82ES4Z1JVpo7ozrJsOOVTtiwa68ltrc0t5+oscPt58gMhQGzeOSefms04jSecaV+2UBrryK8YYVuws4ukvdvB1TiHRESHcNT6D63+SrmPHVbunga78gtNp+HzrQWYvzeG7vaUkRoXxwEV9+PnI7nTU/nGlAA105ePqHE7+t2Efzyzdwdb9h0iNjWDW5QP42fBUvQ+nUg1ooCufVF3n4J01efznyx3sKTpCRueOPDF5MJcOSiZYp6xVqlEa6MqnHKmp47Vv9/LcVzs5UF7NoNRonr12OOf3TdLL8pVqhga68gllR2p5acVu5n6zi5IjtYw6LY7HJw1hTK94HXqolJc00JWlDh6q4oWvd/HKij0crnEwvk9nbj23J8O7x1ldmlJ+RwNdWSK3+Ahzlu3kjexc6hxOLh6UzC/P7km/5E7NP1kp1SgNdNWmcg4e4umlO/jv2nyCBK4alsovzu5Jj4QOVpemlN/TQFdtYoO9jKeX5vDhpv2EBQdx3eh0po/tQddonbpWqZaiga5ajTGGVbuKmb10B8u2FRAVHszt5/bi+p+kE98xzOrylAo4GuiqxRljWPp9AbO/yCF7TwkJHUO5b0IfrhnVjajwEKvLUypgaaCrFuNwGj7YuI/ZX+xgy75yUmIieHhif67OTNOrOpVqAxro6pQ5nIZ31th5eukOdhUe5rTEDjw2aTAThyQTold1KtVmNNDVKVmzt4SH/ruJDXll9E/uxNPThvHT/l2w6VWdSrU5DXR1UgoOVfPoh1t5c7WdpE5hPDllCJcNTtarOpWykAa6OiF1DifzV+7hH59so6rWwS/OPo07xmXoFLZK+QD9K1ReW7GjiJmLN/H9gUOclZHAQ5f2p1fnjlaXpZRy00BXzdpXVslfl2zlvXX5pMRE8J9rhvPT/knavaKUj9FAV02qrnPw4te7+dfn26lzGu4an8EtZ/ckIlSHICrlizTQVaOWfn+Qh9/bzM7Cw5zfL4k/XtyPbvGRVpellDoOrwJdRCYATwI24HljzCMNtj8BnOt+GAl0NsbEtGShqm3kFh/h4fc388nmA/RI6MDcG87g3N6drS5LKeWFZgNdRGzAbOB8wA5kichiY8zmo/sYY+722P8OYGgr1KpaUVWtg2eW7uCZL3cQHCTcN6EPN56ZTliwdq8o5S+8aaGPAHKMMTsBRGQBMBHY3MT+U4GHWqY81dqMMXy06QCz/rcZe0kllw5O5oGL+ugsiEr5IW8CPQXI9XhsB0Y2tqOIdAd6AJ83sX0GMAOgW7duJ1Soank7CiqYuXgTX20vpHdSFK9PH8XonvFWl6WUOkktfVJ0CvCWMcbR2EZjzBxgDkBmZqZp4ddWXqqoruNfn2/nxa93ER5s46FL+3HtqO4E67wrSvk1bwI9D0jzeJzqXteYKcBtp1qUah3GGBavy+evS7ZwoLyaScNTuXdCHxKjdG5ypQKBN4GeBWSISA9cQT4F+HnDnUSkDxALrGjRClWL2LKvnIcWb2LVrmIGpkTzzDXDGdYt1uqylFItqNlAN8bUicjtwEe4hi2+aIzZJCIPA9nGmMXuXacAC4wx2pXiQ8oqa3nik228vGI30REh/PWKgUw+I01nQ1QqAHnVh26MWQIsabDuwQaPZ7ZcWepUOZ2Gt1bb+X8fbqXkSA3TRnbn1xecTkxkqNWlKaVaiV4pGoDW5Zby4OJNrMstJbN7LC9PHEH/5Giry1JKtTIN9ABSVFHN3z/6njeyc0noGMYTkwdz+ZAUnURLqXZCAz0A1DmcvLZqL4999D1HahzcfGYP7hyfoTdkVqqd0UD3c6v3lPCHdzeyZV85Y3rFM/PS/mQkRVldllLKAhrofmxhdi4PvLOBzlFhPDNtGBMGdNHuFaXaMQ10P+R0Gv7xyTb+/UUOZ2UkMHvaMDpp94pS7Z4Gup+pqnXw27fW8966fKackcafLx9AiF6yr5RCA92vFFVUM2P+albvKeH+C/vwi7GnaReLUqqeBrqf2FFQwY3zsthfVsXT04Zx0cCuVpeklPIxGuh+YOXOIn4xfzXBQcLrM0bpHCxKqUZpoPu4d9bYue/t9XSLi2TeDSNIi9P7eiqlGqeB7qOMMTzx6Xae+mw7P+kZzzPThhMdqSNZlFJN00D3QdV1Du57az3vrs1n0vBU/nLFQEKDdSSLUur4NNB9TMnhGn4xfzWrdhfz25/25tZzeupIFqWUVzTQfciuwsPcOC+LvNJK/jV1KJcOTra6JKWUH9FA9xGrdhUzY342QSK8Pn0kw7vHWV2SUsrPaKD7gP+uzeO3b64nNTaCuTecQff4DlaXpJTyQxroFjLG8K/Pc/jHJ9sY0SOOOdcO1zsKKaVOmga6RWrqnNz/znreWZPHlUNT+NtVAwkLtlldllLKj2mgW6D0SA23vLKalTuLufu807lzfC8dyaKUOmUa6G1sT9FhbpiXhb24kicmD+aKoalWl6SUChAa6G1o9Z5ipr+8GqcxzL9pBCNPi7e6JKVUANFAbyPvrcvn12+uIzk6nBevP4PTEjtaXZJSKsBooLcyYwxPL93B3z/6njPSY3n22kziOuhIFqVUy/NqghARmSAi34tIjojc38Q+V4vIZhHZJCKvtWyZ/qmmzsm9b63n7x99z8Qhybxy80gNc6VUq2m2hS4iNmA2cD5gB7JEZLExZrPHPhnA74AxxpgSEencWgX7i7LKWn75ymqW7yjizvEZ3H1eho5kUUq1Km+6XEYAOcaYnQAisgCYCGz22Gc6MNsYUwJgjDnY0oX6k9ziI9wwL4s9RYd5fNJgrhquI1mUUq3Pm0BPAXI9HtuBkQ32OR1ARL4BbMBMY8yHDQ8kIjOAGQDdunU7mXp93pq9JUx/KZtah5OXbxzJ6J46kkUp1TZaapLtYCADOAeYCjwnIjENdzLGzDHGZBpjMhMTE1vopX3Hkg37mDpnJZFhNt65dYyGuVKqTXnTQs8D0jwep7rXebID3xpjaoFdIrINV8BntUiVPs4Yw7PLdvLIB1sZ1i2G5/4vk/iOYVaXpZRqZ7xpoWcBGSLSQ0RCgSnA4gb7vIurdY6IJODqgtnZgnX6rFqHkwcWbeCRD7Zy8aCuvDZ9lIa5UsoSzbbQjTF1InI78BGu/vEXjTGbRORhINsYs9i97QIR2Qw4gN8aY4pas3BfUF5Vy22vrm/fONQAABMFSURBVOGr7YXcek5PfnNBb4KCdCSLUsoaYoyx5IUzMzNNdna2Ja/dEvJKK7lh7ip2Fhzmr1cM5Ooz0pp/klJKnSIRWW2MyWxsm14pepJ+v2gD+aVVzLthBGdmJFhdjlJKtdgol3bFXnKEL7cVcOOZPTTMlVI+QwP9JLyZbQdgkl4wpJTyIRroJ8jhNLyZncuZvRJIi4u0uhyllKqngX6CvtpeQH5ZFVPOCMwrXZVS/ksD/QS9kZVLXIdQzu+XZHUpSil1DA30E1BYUc0nmw9w1bAUQoP1rVNK+RZNpRPwzho7dU7DZB1zrpTyQToO3UvGGBZk5ZLZPZZenaOsLkcp3+N0wJFiOFzg8VXo+l5XBUE2CAr2+GrusTf7nOBzbGEQFLjtWA10L2XtLmFnwWF++bOeVpeiVNswBqrLfwjlhiFdv+x+fKQIaOTKcwmC4AgwDnDWub6sZAuFkAhXTSHhEBIJweHude7v9cuRrn0a29fzGMHux8csR7heqw1vbKOB7qUFWXvpGBbMxYO6Wl2KUievttIjkI8T1EeKXN8dNY0fJzwGOiS6vhJ6QffRPzzukOCxnOja17NVbAwY5w/h7qxzte6PedzYupPZx+Oxo9b189RWur7qKqG2CmqPuD5B1FZBVSkc2ufe7t5WWwWO6pN8w6Xx8B/7G+h/+Ukes2ka6F4oq6xlyYZ9XDkslchQfcvaLUcd1ByC6gqoPgQ1Db5XV/ywvfaIK7TAFWBHW671cyeZ4yzjxT4ey42+hnu5rvrYwK6paPxnC46Aju4A7pQMXQe5liMTfhzSkfEQfAr3xhUBsbm6QvCTmUmdzh/+A/jRfwQN/oOoq/RYV9X4cmjHVilT08kLi9flU1XrZIqeDPUvxrgC7XjBe8z6o8sV7u/lHssVrj9Ub9hCXR/NJcjj47ac4rL7O7jXe7lsC3WFcWyPRlrPCT88Du3g/fvaHgUFud4jH3+fNNC98EbWXvp17cTAlGirSwks9a2eSvdH28a+N7eukX08A9tZ610tIZGuVlNYRwiLgtAo6JTiXhflWh8a5bF8dH2Ux/M6uZZPpfWq1CnQQG/GxrwyNuaV8/DE/kgbntzwypFi2L/e1d959CO4cf7wkdurZXOC+zvdn+YbWV9X7WUIu7+fVL+kuE9URXh8dy+HR0NUlwbB3EjwNgzm0I5g0z8F5f/0t7gZb2TlEhYcxMTBKdYWcrgQ9q2F/LWwb51ruXSvtTU1FBzRIGjdowJCI10f7UMabm8imOtPIDWyLTisTUcNKOVPNNCPo7LGwbtr87hoYFeiI0Pa7oUP7XeFtmd4l3vcxjXuNEgZDpk3QdfBrlapBOHqa/Xot61fFi+X5QT3D/J4jlLKahrox7Fkwz4OVdW13pWhxriCumF4Vxxw7yAQ3wu6/8QV3F2HQJeBEBHTOvUopfyaBvpxvJGVS3p8JCN7xJ36wYyB0j0NwnsdHCl0bZcgSOgNPcd5hPcAV1+vUkp5QQO9CTsKKli1u5j7JvQ58ZOhTieU7GrQ573OddECuC5BTuwLvSe4grvrEEjq7+prVkqpk6SB3oSFWbkEBwlXDW/mZKjTAUU5x7a89693jWEG1zjgzv2g30RIHuJqfXfu7zphqJRSLUgDvRE1dU7eXmNnfN/OdI5qJHidDlg9F9a/Cfs3QO1h1/rgcEgaAAMn/RDeiX11XLJSqk1ooDfi860HKKyoafyuRAe3wOI7wb7KFdhDr3F9Tx7i6gPX8cxKKYto+jRiQVYuXTqFM/b0xB9W1lXDV/+Arx53nai8Yg4MulqH7CmlfIZXEwOLyAQR+V5EckTk/ka2Xy8iBSKy1v11c8uX2jbySiv5clsBV2emYgtyh/Xeb+E/Z8GXj8CAK+H2LBg8WcNcKeVTmm2hi4gNmA2cD9iBLBFZbIzZ3GDXN4wxt7dCjW3qzexcACZlpkFVOXz2MGQ9D9GpMO1tyDjP4gqVUqpx3nS5jAByjDE7AURkATARaBjofs/hNLyZbefMXgmkFXwJ79/jmht55C0w7g+uuT+UUspHedPlkgLkejy2u9c1dJWIrBeRt0Sk0UsrRWSGiGSLSHZBQcFJlNu6vs4ppLp0P//P+Q94fYrrisybP4ULH9EwV0r5vJa6ud57QLoxZhDwCfBSYzsZY+YYYzKNMZmJiYmN7WIdY9jz6bN8Fv4buu7/3NUin/ElpGZaXZlSSnnFmy6XPMCzxZ3qXlfPGFPk8fB54NFTL60NFe+k5t07+b+DX7G34xCir3sOEk+3uiqllDoh3rTQs4AMEekhIqHAFGCx5w4i4nmjzcuALS1XYity1MHX/4SnR0PedzxQexPV1yzWMFdK+aVmW+jGmDoRuR34CLABLxpjNonIw0C2MWYxcKeIXAbUAcXA9a1Yc8vIXwuL74D96zF9Luba3KuoS+hKRhe9K5FSyj95dWGRMWYJsKTBugc9ln8H/K5lS2slNUdg6d9gxWzXTReunk925Jl8u3YFj56r9wxVSvmv9nWl6M6l8N5dULIbhl0H5z8METEsWLiOjmHBXDywa3NHUEopn9U+Av1IMXz8R1j7CsT1hOvehx5nAVBeVcv/NuRzxdBUOoS1j7dDKRWYAjvBjIFNi+CDe12hfuY9cPa9rvtTui1em09VrZMprXVXIqWUaiOBG+hldvjfb2DbB5A8FK5d5Lp9WwNvZOXSp0sUg1L1ZKhSyr8FXqA7nZD9Anw6E4wTLviL69L9Rqa13ZhXxoa8Mv50Wf8TvyuRUkr5mMAK9INb4b07Ifdb1705L3kCYtOb3H1hdi6hwUFcPqSZuxIppZQfCIxAr6uGr5+AZY+55ly54lkYdPzpbatqHSz6Lo+LBnQhOjKkDYtVSqnW4f+BnrvKdYFQwVbXrd9++jfo2Pw8MUs27ONQVR2TG7srkVJK+SH/DfTqQ665ylc9556r/C3ION/rpy/IyiU9PpJRp8W1YpFKKdV2/DPQv/8Q/ncPlOfDyF/AuD+e0PS2OwsqWLWrmHsn9NaToUqpgOF/gf7VP+CzP0FiX7jpJUg744QP8UZ2LrYg4WfDUluhQKWUsob/BXq/ieB0wJi7IDj0hJ9e63Dy9mo74/p0pnOn8FYoUCmlrOF/gR7fE87+7Uk//bMtBymsqGHqCL0yVCkVWFrqjkV+442svXTpFM7YDB+7Y5JSSp2idhXo+aWVfLmtgEmZqQTb2tWPrpRqB9pVqr2Zbcdp4OpM7W5RSgWedhPoDqdhYXYuZ/ZKIC0u0upylFKqxbWbQP8mp5C80kom6zS5SqkA1W4C/Y2sXGIiQ7igf5LVpSilVKtoF4FeVFHNx5v3c+XQVMKCbVaXo5RSraJdBPqi7/KodRim6NhzpVQAC/hAN8awICuXYd1iOD0pyupylFKq1QR8oK/ZW0LOwQqm6DS5SqkAF/CB/vqqXDqE2rh4UFerS1FKqVblVaCLyAQR+V5EckTk/uPsd5WIGBHJbLkST155VS3/W7+Py4Yk0yHM/6atUUqpE9FsoIuIDZgNXAj0A6aKSL9G9osC7gK+bekiT9Z76/KprHXoXYmUUu2CNy30EUCOMWanMaYGWABMbGS/PwP/D6hqwfpOyRtZufTpEsXg1GirS1FKqVbnTaCnALkej+3udfVEZBiQZoz53/EOJCIzRCRbRLILCgpOuNgTsSm/jPX2MiafkaZ3JVJKtQunfFJURIKAfwC/bm5fY8wcY0ymMSYzMbF1p69dmJVLaHAQVwxNaX5npZQKAN4Eeh7geUVOqnvdUVHAAGCpiOwGRgGLrTwxWlXrYNF3eVw4oAsxkSd+VyOllPJH3gR6FpAhIj1EJBSYAiw+utEYU2aMSTDGpBtj0oGVwGXGmOxWqdgLH2zcR3lVnU7EpZRqV5oNdGNMHXA78BGwBVhojNkkIg+LyGWtXeDJWLAql+7xkYzqEW91KUop1Wa8GpxtjFkCLGmw7sEm9j3n1Ms6eTsLKvh2VzG//WlvgoL0ZKhSqv0IuCtFF2bbsQUJPxueanUpSinVpgIq0GsdTt5abefc3p1J6hRudTlKKdWmAirQP996kMKKaqboyVClVDsUUIH+RlYuSZ3COKd3645xV0opXxQwgb6vrJKl3x9k0vA0gm0B82MppZTXAib53sy24zRwdaZ2tyil2qeACHSn0/BGVi5jesXTLT7S6nKUUsoSARHo3+woJK+0UqfJVUq1awER6AuycomJDOGCfklWl6KUUpbx+0AvPlzDx5v2c8XQFMJDbFaXo5RSlvH7QH9njZ1ah9GJuJRS7Z5fB7oxrpOhQ9Ji6NOlk9XlKKWUpfw60NfsLWX7wQqmjtDWuVJK+XWgL1i1lw6hNi4ZlGx1KUopZTm/DfRDVbW8v34flw5OpkOYV7MAK6VUQPPbQH9v3T4qax16MlQppdz8NtDfyNpL76QohqTFWF2KUkr5BL8M9M355ayzlzH5jDRE9K5ESikFfhroC7NzCbUFccXQFKtLUUopn+F3gV5V6+CdNXZ+OqALsR1CrS5HKaV8ht8F+keb9lNeVcdUPRmqlFLH8LtA7xAazAX9khh1WrzVpSillE/xuwHc5/VL4jydVVEppX7E71roSimlGudVoIvIBBH5XkRyROT+RrbfIiIbRGStiHwtIv1avlSllFLH02ygi4gNmA1cCPQDpjYS2K8ZYwYaY4YAjwL/aPFKlVJKHZc3LfQRQI4xZqcxpgZYAEz03MEYU+7xsANgWq5EpZRS3vDmpGgKkOvx2A6MbLiTiNwG3AOEAuMaO5CIzABmAHTrpvf/VEqpltRiJ0WNMbONMT2B+4A/NLHPHGNMpjEmMzExsaVeWimlFN4Feh7geRVPqntdUxYAl59KUUoppU6cN4GeBWSISA8RCQWmAIs9dxCRDI+HFwPbW65EpZRS3mi2D90YUycitwMfATbgRWPMJhF5GMg2xiwGbheR84BaoAS4rrnjrl69ulBE9pxk3QlA4Uk+NxDp+3EsfT9+oO/FsQLh/eje1AYxxv8GpIhItjEm0+o6fIW+H8fS9+MH+l4cK9DfD71SVCmlAoQGulJKBQh/DfQ5VhfgY/T9OJa+Hz/Q9+JYAf1++GUfulJKqR/z1xa6UkqpBjTQlVIqQPhdoDc3lW97ISJpIvKFiGwWkU0icpfVNfkCEbGJyHci8r7VtVhNRGJE5C0R2SoiW0RktNU1WUVE7nb/nWwUkddFJNzqmlqDXwW6l1P5thd1wK+NMf2AUcBt7fi98HQXsMXqInzEk8CHxpg+wGDa6fsiIinAnUCmMWYArgskp1hbVevwq0DHi6l82wtjzD5jzBr38iFcf6wp1lZlLRFJxTX1xPNW12I1EYkGxgIvABhjaowxpdZWZalgIEJEgoFIIN/ielqFvwV6Y1P5tusQAxCRdGAo8K21lVjun8C9gNPqQnxAD6AAmOvugnpeRDpYXZQVjDF5wGPAXmAfUGaM+djaqlqHvwW6akBEOgJvA79qcKORdkVELgEOGmNWW12LjwgGhgHPGGOGAoeBdnnOSURicX2S7wEkAx1E5Bprq2od/hboJzqVb0ATkRBcYf6qMeYdq+ux2BjgMhHZjasrbpyIvGJtSZayA3ZjzNFPbW/hCvj26DxglzGmwBhTC7wD/MTimlqFvwV6s1P5thciIrj6R7cYY9r9PVyNMb8zxqQaY9Jx/V58bowJyFaYN4wx+4FcEentXjUe2GxhSVbaC4wSkUj33814AvQEsTe3oPMZTU3la3FZVhkDXAtsEJG17nUPGGOWWFiT8i13AK+6Gz87gRssrscSxphvReQtYA2u0WHfEaBTAOil/0opFSD8rctFKaVUEzTQlVIqQGigK6VUgNBAV0qpAKGBrpRSAUIDXQUcEXGIyFqPrxa7QlJE0kVkY0sdT6mW5Ffj0JXyUqUxZojVRSjV1rSFrtoNEdktIo+KyAYRWSUivdzr00XkcxFZLyKfiUg39/okEVkkIuvcX0cvF7eJyHPu+bU/FpEI9/53uuenXy8iCyz6MVU7poGuAlFEgy6XyR7byowxA4F/45qdEeBfwEvGmEHAq8BT7vVPAV8aYwbjmgfl6FXJGcBsY0x/oBS4yr3+fmCo+zi3tNYPp1RT9EpRFXBEpMIY07GR9buBccaYne6JzfYbY+JFpBDoaoypda/fZ4xJEJECINUYU+1xjHTgE2NMhvvxfUCIMWaWiHwIVADvAu8aYypa+UdV6hjaQlftjWli+URUeyw7+OFc1MW47qg1DMhy30xBqTajga7am8ke31e4l5fzwy3JpgFfuZc/A34J9fcqjW7qoCISBKQZY74A7gOigR99SlCqNWkLQgWiCI8ZKMF1X82jQxdjRWQ9rlb2VPe6O3Dd2ee3uO7yc3RWwruAOSJyE66W+C9x3fGmMTbgFXfoC/BUO7/lm7KA9qGrdsPdh55pjCm0uhalWoN2uSilVIDQFrpSSgUIbaErpVSA0EBXSqkAoYGulFIBQgNdKaUChAa6UkoFiP8P73qW0DrQEVcAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 573
        },
        "id": "5RUG84umHLtB",
        "outputId": "a6991b79-333a-4601-8963-b980c6113590"
      },
      "source": [
        "plot_loss_curves(history_model_0_fine_tune)"
      ],
      "execution_count": 23,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEWCAYAAAB2X2wCAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAfNElEQVR4nO3de3Rcdb338fc3M5N7LzEtTa8EtHKtFok9IKAIHgREQAQqIEuKyHoq98NCOD4oysIlR1we5Yh04Tk8UkQBARUVlKtP7bIgaW0pFSzQh0t6oWmgbe6ZJN/nj72TTJJJMkknmWTn81pr1r7v/dvp6uf3m9++jLk7IiIy8eXlugAiIpIdCnQRkYhQoIuIRIQCXUQkIhToIiIRoUAXEYkIBbqISEQo0GVSMLM3zOxTuS6HyGhSoIuIRIQCXSYtMyswsx+a2bbw80MzKwiXzTCz35vZbjN718z+YmZ54bLrzWyrmdWb2T/N7MTcnolIIJ7rAojk0P8GjgIWAw78FrgR+AZwLVADzAzXPQpwMzsIuBz4qLtvM7NKIDa2xRZJTy10mcwuAG52953uXgt8G7gwXJYEZgP7u3vS3f/iwYuPOoAC4FAzS7j7G+7+ek5KL9KHAl0msznAmynTb4bzAG4DXgOeMLMtZnYDgLu/BlwNfAvYaWb3m9kcRMYBBbpMZtuA/VOmF4TzcPd6d7/W3Q8ETgf+rauv3N1/4e7Hhts68B9jW2yR9BToMpkkzKyw6wP8ErjRzGaa2Qzgm8DPAczsNDP7gJkZsIegq6XTzA4ysxPCi6ctQDPQmZvTEelNgS6TyWMEAdz1KQSqgReBjcA64JZw3YXAU0ADsAb4ibs/S9B/fiuwC9gB7Af8+9idgsjATD9wISISDWqhi4hEhAJdRCQiFOgiIhGhQBcRiYicPfo/Y8YMr6yszNXhRUQmpLVr1+5y95npluUs0CsrK6murs7V4UVEJiQze3OgZepyERGJCAW6iEhEKNBFRCJCgS4iEhEKdBGRiFCgi4hEhAJdRCQi9JuiIiKjqb0N6rfBnprw8zbMPRLef0LWD6VAFxEZKXdoejcI6dTA3ru1Z7p+B8EPW6U45moFuojImEq29A7nrsBOnW5v7r1NrACmzQs+7z+xZ3zaPJg2H6bNhUTRqBRXgS4ik5M7NO7qH9Cp0407+29XOisI51mHwsKT+gd2yQwwG/vzQYEuIlGVbIY9WwcP7I7W3tvEi2D6/CCcKw4PW9QpgT11LsQLcnM+GVCgi8jE09kJjbVpukBSxpt29dnIYEpFEMyzPwQHn9onsOdDUVnOWtfZoEAXkfGnrTFNWKe0tvduhY623tskSnpa13MWp/RZh4E9ZQ7E83NzPmNEgS4iYyvZAvXbg8/ebX0uOoaB3fxe720sLwjkaXNh7kfg0NP7d4cUTp/QretsUKCLSHa4Q1NdENJdYd1rGIZ487v9ty2Y1hPM85akaV3Phpjiaij6C4nI0JItwcMx9Tv6BPS2nmH9jv7dIBiUzISps2H6AljwL0FLe+rsIKSnzgk+hdNyclpRo0AXmcwyalVv698FApAo7gnl+UeFIT2nZzilIvjEEmN/XpOUAl0kqrpa1V1dHcNpVZfuF4R12f6w4KgwuFNa1VNmB63qSd5nPd4o0EUmmtFsVU+dHTw4o1b1hKRAFxlP+raqe4X1juG1qtOFdcFUtaojTIEu0ldnZ/AEYXtLELDtLdDe2mc4yPwBt2kN3vuRdn7Xds39y5Mo6enuWHB0SrdHhVrV0osCXcYnd2hrGGZgDhKkyUGCtO92/Vq/w2R5wSPk8QKIF/YeJoqCT1FZn+Xh+oVT1aqWERsy0M1sPrASmEXwDsi73P1HfdYx4EfAqUATcJG7r8t+cWXC63rdaMOO8ELdO+FwRzhvRzCvIV23wjDE8vuEaWHv6cJpvUM2XfgOFMrxQkik2WfXUC1lyZFMWujtwLXuvs7MpgBrzexJd/9HyjqnAAvDz78Ad4ZDmSz6BfWOnk9D6vg76YO6cHrPbW77fywYFpcPELYDBGlXyMYKIE8/xiWTz5CB7u7bge3heL2ZvQzMBVID/Qxgpbs78JyZTTez2eG2MpFlNaiPCcdnw5RZwbB0VjBvlN4PLTKZDKsP3cwqgSOA5/ssmgu8nTJdE85ToI9X2QzqymPDYE4J6ikVwTwFtciYyTjQzawUeBi42t33juRgZnYpcCnAggULRrILGUpXUNdv7x3M9WFwN7yjoBaJqIwC3cwSBGF+n7s/kmaVrcD8lOl54bxe3P0u4C6Aqqoq77tcBqGgFpEhZHKXiwH/A7zs7j8YYLVHgcvN7H6Ci6F7JmX/eWdHcHtcsil4n3O/8UZoa+oZTzaHy5qGHm+th85k/2MqqEUklEkL/RjgQmCjma0P530dWADg7iuAxwhuWXyN4LbFZdkvahZ0dgwRnk1h4A403hXCA6zT3jL8MiWKg09+ccp4SfDUX9d4oggKpkBphYJaRAaUyV0uq4FBn2oI7265LFuFGlTd6/DaU5m1gPu2hkczcAccLwm3TR3v+hTpgRERyZqJ96Tojo3w+NfCCesJxvziIDATRWHgVvSMd6+TbrxvWKfsR4ErIhPIxAv0D34artuiwBUR6WPiBXpXkIuISC96PlpEJCIU6CIiEaFAFxGJCAW6iEhEKNBFRCJCgS4iEhEKdBGRiFCgi4hEhAJdRCQiFOgiIhGhQBcRiQgFuohIRCjQRUQiQoEuIhIRCnQRkYhQoIuIRIQCXUQkIhToIiIRoUAXEYkIBbqISEQo0EVEIkKBLiISEQp0EZGIUKCLiESEAl1EJCIU6CIiEaFAFxGJCAW6iEhEKNBFRCJCgS4iEhEKdBGRiFCgi4hEhAJdRCQiFOgiIhGhQBcRiYghA93M7jaznWb20gDLjzezPWa2Pvx8M/vFFBGRocQzWOdnwI+BlYOs8xd3Py0rJRIRkREZsoXu7quAd8egLCIisg+y1Yd+tJltMLPHzeywgVYys0vNrNrMqmtra7N0aBERgewE+jpgf3f/MPBfwG8GWtHd73L3KnevmjlzZhYOLSIiXfY50N19r7s3hOOPAQkzm7HPJRMRkWHZ50A3swozs3B8SbjPun3dr4iIDM+Qd7mY2S+B44EZZlYD3AQkANx9BXA2sNzM2oFm4Avu7qNWYhERSWvIQHf384ZY/mOC2xpFRCSH9KSoiEhEKNBFRCJCgS4iEhEKdBGRiFCgi4hEhAJdRCQiFOgiIhGhQBcRiQgFuohIRCjQRUQiQoEuIhIRmfwEnYhMIslkkpqaGlpaWnJdlEmtsLCQefPmkUgkMt5GgS4ivdTU1DBlyhQqKysJ34wtY8zdqauro6amhgMOOCDj7dTlIiK9tLS0UF5erjDPITOjvLx82N+SFOgi0o/CPPdG8m+gQBcRiQgFuoiMK7t37+YnP/nJsLc79dRT2b1796DrfPOb3+Spp54aadHSKi0tzer+9oUCXUTGlYECvb29fdDtHnvsMaZPnz7oOjfffDOf+tSn9ql845nuchGRAX37d5v4x7a9Wd3noXOmctNnDxtw+Q033MDrr7/O4sWLSSQSFBYWUlZWxiuvvMLmzZs588wzefvtt2lpaeGqq67i0ksvBaCyspLq6moaGho45ZRTOPbYY/nrX//K3Llz+e1vf0tRUREXXXQRp512GmeffTaVlZV86Utf4ne/+x3JZJJf/epXHHzwwdTW1nL++eezbds2jj76aJ588knWrl3LjBkzBj0vd+drX/sajz/+OGbGjTfeyNKlS9m+fTtLly5l7969tLe3c+edd/Kxj32ML3/5y1RXV2NmXHzxxVxzzTX7/LdVC11ExpVbb72V97///axfv57bbruNdevW8aMf/YjNmzcDcPfdd7N27Vqqq6u5/fbbqaur67ePV199lcsuu4xNmzYxffp0Hn744bTHmjFjBuvWrWP58uV8//vfB+Db3/42J5xwAps2beLss8/mrbfeyqjcjzzyCOvXr2fDhg089dRTXHfddWzfvp1f/OIXfPrTn+5etnjxYtavX8/WrVt56aWX2LhxI8uWLRvhX6s3tdBFZECDtaTHypIlS3rdi3377bfz61//GoC3336bV199lfLy8l7bHHDAASxevBiAI488kjfeeCPtvs8666zudR555BEAVq9e3b3/k08+mbKysozKuXr1as477zxisRizZs3iE5/4BC+88AIf/ehHufjii0kmk5x55pksXryYAw88kC1btnDFFVfwmc98hpNOOinzP8gg1EIXkXGtpKSke/zPf/4zTz31FGvWrGHDhg0cccQRae/VLigo6B6PxWID9r93rTfYOvvq4x//OKtWrWLu3LlcdNFFrFy5krKyMjZs2MDxxx/PihUruOSSS7JyLAW6iIwrU6ZMob6+Pu2yPXv2UFZWRnFxMa+88grPPfdc1o9/zDHH8OCDDwLwxBNP8N5772W03XHHHccDDzxAR0cHtbW1rFq1iiVLlvDmm28ya9YsvvKVr3DJJZewbt06du3aRWdnJ5///Oe55ZZbWLduXVbKri4XERlXysvLOeaYYzj88MMpKipi1qxZ3ctOPvlkVqxYwSGHHMJBBx3EUUcdlfXj33TTTZx33nnce++9HH300VRUVDBlypQht/vc5z7HmjVr+PCHP4yZ8b3vfY+KigruuecebrvtNhKJBKWlpaxcuZKtW7eybNkyOjs7Afjud7+blbKbu2dlR8NVVVXl1dXVOTm2iAzs5Zdf5pBDDsl1MXKmtbWVWCxGPB5nzZo1LF++nPXr1+ekLOn+LcxsrbtXpVtfLXQRkRRvvfUW5557Lp2dneTn5/PTn/4010XKmAJdRCTFwoUL+fvf/95rXl1dHSeeeGK/dZ9++ul+d9jkkgJdRGQI5eXlOet2GQ7d5SIiEhEKdBGRiFCgi4hEhAJdRCQiFOgiMqEN9j7yN954g8MPP3wMS5NbCnQRkYjQbYsiMrDHb4AdG7O7z4pFcMqtAy6+4YYbmD9/PpdddhkA3/rWt4jH4zz77LO89957JJNJbrnlFs4444xhHbalpYXly5dTXV1NPB7nBz/4AZ/85CfZtGkTy5Yto62tjc7OTh5++GHmzJnDueeeS01NDR0dHXzjG99g6dKl+3TaY0GBLiLjytKlS7n66qu7A/3BBx/kT3/6E1deeSVTp05l165dHHXUUZx++unD+iHlO+64AzNj48aNvPLKK5x00kls3ryZFStWcNVVV3HBBRfQ1tZGR0cHjz32GHPmzOEPf/gDELwUbCJQoIvIwAZpSY+WI444gp07d7Jt2zZqa2spKyujoqKCa665hlWrVpGXl8fWrVt55513qKioyHi/q1ev5oorrgDg4IMPZv/992fz5s0cffTRfOc736GmpoazzjqLhQsXsmjRIq699lquv/56TjvtNI477rjROt2sGrIP3czuNrOdZvbSAMvNzG43s9fM7EUz+0j2iykik8k555zDQw89xAMPPMDSpUu57777qK2tZe3ataxfv55Zs2alfQ/6SJx//vk8+uijFBUVceqpp/LMM8/wwQ9+kHXr1rFo0SJuvPFGbr755qwca7RlclH0Z8DJgyw/BVgYfi4F7tz3YonIZLZ06VLuv/9+HnroIc455xz27NnDfvvtRyKR4Nlnn+XNN98c9j6PO+447rvvPgA2b97MW2+9xUEHHcSWLVs48MADufLKKznjjDN48cUX2bZtG8XFxXzxi1/kuuuuy9r7ykfbkF0u7r7KzCoHWeUMYKUH7+F9zsymm9lsd9+epTKKyCRz2GGHUV9fz9y5c5k9ezYXXHABn/3sZ1m0aBFVVVUcfPDBw97nV7/6VZYvX86iRYuIx+P87Gc/o6CggAcffJB7772XRCJBRUUFX//613nhhRe47rrryMvLI5FIcOedE6OdmtH70MNA/72797uh08x+D9zq7qvD6aeB692938vOzexSglY8CxYsOHIktayIjK7J/j708WS470Mf0/vQ3f0ud69y96qZM2eO5aFFRCIvG3e5bAXmp0zPC+eJiIyJjRs3cuGFF/aaV1BQwPPPP5+jEuVGNgL9UeByM7sf+Bdgj/rPRSY2dx/WPd65tmjRognxvvLhGMnPgw4Z6Gb2S+B4YIaZ1QA3AYnwgCuAx4BTgdeAJmDZsEshIuNGYWEhdXV1lJeXT6hQjxJ3p66ujsLCwmFtl8ldLucNsdyBy4Z1VBEZt+bNm0dNTQ21tbW5LsqkVlhYyLx584a1jZ4UFZFeEokEBxxwQK6LISOgty2KiESEAl1EJCIU6CIiEaFAFxGJCAW6iEhEKNBFRCJCgS4iEhEKdBGRiFCgi4hEhAJdRCQiFOgiIhGhQBcRiQgFuohIRCjQRUQiQoEuIhIRCnQRkYhQoIuIRIQCXUQkIhToIiIRoUAXEYkIBbqISEQo0EVEIkKBLiISEQp0EZGIUKCLiESEAl1EJCIU6CIiEaFAFxGJCAW6iEhEKNBFRCJCgS4iEhEKdBGRiFCgi4hEhAJdRCQiFOgiIhGhQBcRiQgFuohIRGQU6GZ2spn908xeM7Mb0iy/yMxqzWx9+Lkk+0UVEZHBxIdawcxiwB3AvwI1wAtm9qi7/6PPqg+4++WjUEYREclAJi30JcBr7r7F3duA+4EzRrdYIiIyXJkE+lzg7ZTpmnBeX583sxfN7CEzm59uR2Z2qZlVm1l1bW3tCIorIiIDydZF0d8Ble7+IeBJ4J50K7n7Xe5e5e5VM2fOzNKhRUQEMgv0rUBqi3teOK+bu9e5e2s4+d/AkdkpnoiIZCqTQH8BWGhmB5hZPvAF4NHUFcxsdsrk6cDL2SuiiIhkYsi7XNy93cwuB/4ExIC73X2Tmd0MVLv7o8CVZnY60A68C1w0imUWEZE0zN1zcuCqqiqvrq7OybFFRCYqM1vr7lXplulJURGRiFCgi4hEhAJdRCQiFOgiIhGhQBcRiQgFuohIRCjQRUQiQoEuIhIRCnQRkYhQoIuIRIQCXUQkIhToIiIRoUAXEYkIBbqISEQo0EVEIkKBLiISEUP+YtF4U9fQypZdjZSX5FNeWsDUwjhmlutiiYjk3IQL9DVb6rj8F3/vnk7EjPKSAspLg4CfUZLfPV7eNR4un1FaQGEilsPSi4iMngkX6EcdWM7Ki5dQ19hKXUMbuxraqGtopa4xGL6+s4G6xlZakp1pty/JjwVhHwb9jNL+od81XVacIB5Tr5SITAwTLtBnlBbw8Q/OHHQdd6eprYN3G9vY1RAEf11jaxj+bd2VQc17TWyo2c27jW10dPb/bVUzmF6U6G7tp4Z9EP6p3wTU/SMiuTXhAj0TZkZJQZySgjjz31c85Pqdnc6e5mR3K79ruCsl/Osa2nh5x17qGtrY05xMu59EzHhfSZ/Wfhj23RWAun9EZJREMtCHKy/PKCvJp6wknw/sVzrk+m3tnbzX1Lu1vyulIqhraGNXYxv/b1cjuxqG7v55X0nvsC8vLegznc/7ivPV/SMig1Kgj0B+PI9ZUwuZNbUwo/Wb2tp7Qr+hLegK6m75BxXB1t0tvFizh3cb22hP0/0DQQUwtSjB1MIE04oSTC2KM7UwEc6LB8NwedeyaeG8KQVx8vLUHSQSZQr0MVCcH6f4fZl3/+xtSfa+2NvYxrsNbextSbK3OcneliR7mpNs293CKy317G1OUt/ajqevB4DgekBpQTxtZTCtTyXQPS9luiQ/pusDIuOcAn2cycszphfnM704s+6fLp2dTn1re3fg721u7w7+YF7qsmD5W+82dS9raG0fdP+xPOv5FpDmG0Dqsv7fHhIUJvJUIYiMMgV6ROTlGdPClvVItHd0Ut/SPkBlkL6C2Lm3oXtZc7Jj0P3nx/K6Q35KWAFMG6SLaGphnCmFCUoL4hQXxCjJjxNTl5HIoBToAkA8ltd9YXgkWts7ggoh5dvAnj6VQc+8YPnW3c3BsuYkbR3pLxynKkzkURrevVScH6e0IBYO45T0Gg+mS/LDYde8/NT5cfLjusgs0aJAl6woiMcoKI0xo7Rg2Nu6O63tnSnXB3q6h5raOmhsbaextYPGtqBrqKm1nYbWYP7upuB5gq7lja3tDHBNuZ/8WF5367/rm0BpQZzi/KASKE2pOHoqhJ71UiuM0oI4BXF1K0luKdAl58yMwkSMwkSM/TK8c2ggXZVDQ2t7moogrBzC4G9o7aApXNbY2k5TWwcNre28s7ele7um1o6Mvj1AcJ2hOL/nm0FXhdD9jaG7koilfNOIpVQcQWVRlAg/+TFVEjIsCnSJlNTKYSTfFtJpa+9MqQiC0G9qS19h9K0cGlvb2bq7OZwXrDfQcwkD6Qr3okSMwkReyngwLM4PlndNd61fmDLevX5+zzqF+Xnd43rGIRoU6CJDyI/nkR8f+fWFvto7OmlKdnUlhZVCazuNbcE3hua2DpqTwaclZby5rZOW7vHgmkVtfWv3dHOyg5ZkB8mODPucUiRiNkT4D1BZhBVMauVRPEBlom8bo0+BLjLG4rE8psbymFo4sjuShpLs6An+lrbOlAqho1eF0NSvwui9vKuC2NuSDJd19lo2XGb0VBB9vnUUxIPA7wr+gpR5wXSs9zB13XgsXD8Y795fOC8/NnkqEgW6SMQkYnkkYnlMGaUKA3quVTRnUCEE4500t7X3+7bR1NYe7CfZwe7mNlqTnbS2B8ta2ztpbQ+Ggz00NxQzeoK/T2VRmFJBpFYCvSqLcJt083pVICn7KAyH+bG8MX1CW4EuIsOWeq2ibJSP5e4kO5yW9o4w8MOwT3b2n5daGSR75rWmVhBpKo2G1vbu/bSk7K8l2ZHxXVMDyY/l9a5IEnmcv2QBlxx3YHb+QCkU6CIyrpkZ+XELnhvYt5ugRqS9o5OWvhVEV/Cnm5dSafStILoqjWxdsO9LgS4iMoh4LI/SWPBQ23ine5VERCIio0A3s5PN7J9m9pqZ3ZBmeYGZPRAuf97MKrNdUBERGdyQgW5mMeAO4BTgUOA8Mzu0z2pfBt5z9w8A/wn8R7YLKiIig8ukhb4EeM3dt7h7G3A/cEafdc4A7gnHHwJOtMly46eIyDiRSaDPBd5Oma4J56Vdx93bgT1Aed8dmdmlZlZtZtW1tbUjK7GIiKQ1phdF3f0ud69y96qZM2eO5aFFRCIvk0DfCsxPmZ4Xzku7jpnFgWlAXTYKKCIimckk0F8AFprZAWaWD3wBeLTPOo8CXwrHzwaecd+Xh3VFRGS4LJPcNbNTgR8CMeBud/+Omd0MVLv7o2ZWCNwLHAG8C3zB3bcMsc9a4M0RlnsGsGuE205UOufJQec8OezLOe/v7mn7rDMK9PHGzKrdvSrX5RhLOufJQec8OYzWOetJURGRiFCgi4hExEQN9LtyXYAc0DlPDjrnyWFUznlC9qGLiEh/E7WFLiIifSjQRUQiYsIF+lCv8o0aM7vbzHaa2Uu5LstYMbP5Zvasmf3DzDaZ2VW5LtNoM7NCM/ubmW0Iz/nbuS7TWDCzmJn93cx+n+uyjAUze8PMNprZejOrzvr+J1Ifevgq383AvxK8JOwF4Dx3/0dOCzaKzOzjQAOw0t0Pz3V5xoKZzQZmu/s6M5sCrAXOjPi/swEl7t5gZglgNXCVuz+X46KNKjP7N6AKmOrup+W6PKPNzN4Aqtx9VB6kmmgt9Exe5Rsp7r6K4OnbScPdt7v7unC8HniZ/m/4jBQPNISTifAzcVpbI2Bm84DPAP+d67JExUQL9Exe5SsREv761RHA87ktyegLux/WAzuBJ9096uf8Q+BrQGeuCzKGHHjCzNaa2aXZ3vlEC3SZRMysFHgYuNrd9+a6PKPN3TvcfTHBG02XmFlku9jM7DRgp7uvzXVZxtix7v4Rgl+AuyzsUs2aiRbombzKVyIg7Ed+GLjP3R/JdXnGkrvvBp4FTs51WUbRMcDpYZ/y/cAJZvbz3BZp9Ln71nC4E/g1QTdy1ky0QM/kVb4ywYUXCP8HeNndf5Dr8owFM5tpZtPD8SKCC/+v5LZUo8fd/93d57l7JcH/42fc/Ys5LtaoMrOS8CI/ZlYCnARk9e61CRXo4c/bXQ78ieBC2YPuvim3pRpdZvZLYA1wkJnVmNmXc12mMXAMcCFBq219+Dk114UaZbOBZ83sRYKGy5PuPilu5ZtEZgGrzWwD8DfgD+7+x2weYELdtigiIgObUC10EREZmAJdRCQiFOgiIhGhQBcRiQgFuohIRCjQJXLMrCPldsf12Xwrp5lVTqY3X8rEEs91AURGQXP4CL3IpKIWukwa4buovxe+j/pvZvaBcH6lmT1jZi+a2dNmtiCcP8vMfh2+o3yDmX0s3FXMzH4avrf8ifDJTszsyvAd7i+a2f05Ok2ZxBToEkVFfbpclqYs2+Pui4AfE7ztD+C/gHvc/UPAfcDt4fzbgf/r7h8GPgJ0PZW8ELjD3Q8DdgOfD+ffABwR7ud/jdbJiQxET4pK5JhZg7uXppn/BnCCu28JX/61w93LzWwXwQ9qJMP52919hpnVAvPcvTVlH5UEj+UvDKevBxLufouZ/ZHgx0h+A/wm5f3mImNCLXSZbHyA8eFoTRnvoOda1GeAOwha8y+Yma5RyZhSoMtkszRluCYc/yvBG/8ALgD+Eo4/DSyH7h+fmDbQTs0sD5jv7s8C1wPTgH7fEkRGk1oQEkVF4S//dPmju3fdulgWvtGwFTgvnHcF8H/M7DqgFlgWzr8KuCt8w2UHQbhvH+CYMeDnYegbcHv4XnORMaM+dJk0RvsHekVyTV0uIiIRoRa6iEhEqIUuIhIRCnQRkYhQoIuIRIQCXUQkIhToIiIR8f8BD3v/ZO3dDC0AAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEWCAYAAAB2X2wCAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3de5RU5Z3u8e+vq6qvXLqBlrs2k2BERCB0jBNyIqPjGTREczSIRB01UZeZeInOJKKTKDFkJisxJ4ln1CMaMSQalpdgjMfBKw5xxMQmEi94I0alAaVFurGF7q6q/p0/and39b2A6i5q9/NZq1bt/e537/1W9eqn3np31Vvm7oiISP4ryHUDREQkOxToIiIhoUAXEQkJBbqISEgo0EVEQkKBLiISEgp0EZGQUKBL3jGzp8xsl5kV5botIgcTBbrkFTOrAv4H4MApg3je6GCdS2R/KdAl3/wj8CxwJ3BuW6GZTTaz35hZnZntNLP/SNt2oZm9YmYfmtkmM/tkUO5m9vG0enea2bJgeZ6Z1ZrZVWb2LrDCzCrM7KHgHLuC5Ulp+48ysxVmti3Y/kBQ/pKZfSGtXszM3jez2QP2LMmQpECXfPOPwF3B7R/MbKyZRYCHgLeBKmAisArAzBYCS4P9RpDq1e/M8FzjgFHAYcBFpP5fVgTrhwJ7gf9Iq/9LoBSYDhwC/CQoXwmcnVbvZGC7uz+fYTtEMmKay0XyhZl9FlgLjHf3983sVeBWUj32B4PyRJd9HgEedvef9XA8B6a6++Zg/U6g1t2/bWbzgEeBEe7e1Et7ZgFr3b3CzMYDW4HR7r6rS70JwGvARHffbWb3AX909x/u95Mh0gP10CWfnAs86u7vB+t3B2WTgbe7hnlgMvCX/TxfXXqYm1mpmd1qZm+b2W5gHVAevEOYDHzQNcwB3H0b8N/A6WZWDpxE6h2GSFbpQo/kBTMrAc4AIsGYNkARUA68BxxqZtEeQn0L8LFeDruH1BBJm3FAbdp617ev/wx8Avi0u78b9NCfByw4zygzK3f3+h7O9QvgAlL/c+vdfWvvj1Zk/6iHLvnii0ASOBKYFdymAb8Ptm0HfmBmZWZWbGZzg/1uB/7FzOZYysfN7LBg20bgy2YWMbP5wHH9tGE4qXHzejMbBVzXtsHdtwP/CdwcXDyNmdnn0vZ9APgkcDmpMXWRrFOgS744F1jh7u+4+7ttN1IXJRcDXwA+DrxDqpe9CMDd7wW+T2p45kNSwToqOOblwX71wFnBtr78FCgB3ic1br+my/ZzgDjwKrAD+EbbBnffC9wPTAF+s4+PXSQjuigqMkjM7FrgcHc/u9/KIvtBY+gigyAYovkqqV68yIDQkIvIADOzC0ldNP1Pd1+X6/ZIeGnIRUQkJNRDFxEJiZyNoY8ZM8arqqpydXoRkby0YcOG9929sqdtOQv0qqoqampqcnV6EZG8ZGZv97ZNQy4iIiGhQBcRCQkFuohISCjQRURCQoEuIhIS/Qa6md1hZjvM7KVetpuZ3Whmm83shbaf9xIRkcGVSQ/9TmB+H9tPAqYGt4uAWw68WSIisq/6/Ry6u68Lfmm9N6cCKz01h8CzZlZuZuOD+aFFRPJastWJJ1tpSbbSkmglnmwlnnBakklaEh3b4on0OkF5UBYPbql154QjDmHm5PKstzUbXyyaSGrioTa1QVm3QDezi0j14jn00EOzcGoRyWfuTjzp7YEYT7bS3BaaQSh2rHcEakvSO8I1PTiDoI2nbW9JO17vx0r2GMItiVZaB2C6q0OGFx20gZ4xd18OLAeorq7WrGAiWebuJIIeZSrcUsGUSHqnnmIqrJxEa8dyvNN277TcFn6J1s5B2hbGibR9WhI9HyM9SNvbmMx+DEQKjFjEKIwUUBgtIJZ237ZcGDEKowWUFUWD9YLUPmn1irrs07a9MCiLpR2n87G77NPl/LGIYWZZf9yQnUDfSuoHcttMCspEhrR4spWPmhN81JJM3Tcn2NOSpLE5wZ6WBB81Jztt39OSoDme/ha9eyCmh2Ui6F12DeKBEi2w9kBqC6hoJFVWmBZWsUgBpYXR9uVYEHLRAmtfbtsWbQ9O6xS8nUPQegjKYD1qwX3H9kjBwIRlPshGoD8IXGJmq4BPAw0aP5d809rqfNSSCtxU+Cb5qCXRHrh7mhNBELeFcII9zcmOsra6wX57mpO0JFszPn9ZYYSSwiglhUGIFaTCqq1XVxKLMKI4SjTSORC7hmUq6KxTb7B9OdqxLRrp6D129C6NaEFBcJzO26IFRsEQDsp80W+gm9mvgXnAGDOrJfXDuDEAd/+/wMPAycBmUr+ifv5ANVYEUsMKTfHWLqHaJXA79Xw7Qvij5mRQNxn0klNle+PJjM9fFLxVLy2MMCztfuzwYkqL2sqilBVGKCuKUlYU3Bem6qbKOraXxCIKS8mKTD7lsrif7Q58PWstkiGhKZ6kfk+cDz5qoX5PCx/saWHXnjj1H6WW6/fE2b033h7CbT3gPcFypheqIgVGWVvwpoXoxPLCtKBNC9w+Ark0CORYRN/Hk4OTflNUDoi7szeeDII5FdC79rR0Cet46j6os2tPC3taeu8RDyuKUlEWY0RxjLKiKGOGFXJYUWn3wC2KUFbYOXA7ylLhWxQtGLALUCIHGwW6tHN3PmxOUP9RPOgxB4H8UedA7hTae1poSfQ+VjyyJEZFaYyKskLGjijmE+OGM6q0kIqyQipKC9u3VZQWUlEWo7ykkMKoesAi+0OBHlKtrc7uprbwjbOrSwi3BfSu9t5z6j7Ry1hGgUF5aSHlpTFGlRYyqaKUoyfFgiBOBXN5aSGjguWK0kJGlsSIanhCZNAo0PNAa6unhXC8x57zrmAooy206/e09DrOHC2wIHxTIfw3Y4Yx57AgkNtCu6ywU0CPKI7pwp3IQU6BnmPuzs6PWni3oYlt9XvZ3tDEtoa9bK9vYnvDXrbVN/He7qZee86F0YL2HnFFaSHTxo3oEsipoK5oC+uyGMOLohpXFgkhBfoAcnd2702kAjoI5+1BWKfKmtje0NRtDDoWMcaNLGb8yBI+VVXB+PISDhleFPSWC4OwToV2SSyicBYRQIF+QBqbE2yv38u2hqZO96mgTt13/TRHpMAYO7yI8eUlzJg4kn+YPo5xI4qZUJ4K8PHlxYwpK9LwhojsMwV6L5riyY4hkPq9qSGRhs497A+bEp32MYPKYamwnnrIcD53eCUTgpAeP7KECeXFVA4r0oVCERkQQzLQWxKtvLe79zHr7Q172bUn3m2/0WWFjBtZzORRpXz6b0a1h/T4kSWMH1nM2BHF+sidiORM6AI9kWxlx4fNncK5fey6oYlt9U2839jcbb8RxVEmlKeCedah5UwY2TEEMmFkCeNGFlMci+TgEYmIZCbvAr1hT5y/7vyo25j1tobUsMh7u5u6fVyvrDDC+CCsp40b0R7S49N612VFefdUiIh0kncp9qs/vM2PHnmtfb0oWtDes/7Mx8YwobyYcSM7B/aIYn1MT0TCL+8Cff5R4zh87HDGjyxmQnkJFaUxhbWICHkY6B+rHMbHKofluhkiIgcdfSRDRCQkFOgiIiGhQBcRCQkFuohISCjQRURCQoEuIhISCnQRkZBQoIuIhIQCXUQkJBToIiIhoUAXEQkJBbqISEgo0EVEQkKBLiISEgp0EZGQUKCLiISEAl1EJCQU6CIiIaFAFxEJCQW6iEhIZBToZjbfzF4zs81mtqSH7YeZ2RNm9oKZPWVmk7LfVBER6Uu/gW5mEeAm4CTgSGCxmR3ZpdoNwEp3Pxq4Hvj3bDdURET6lkkP/Rhgs7u/6e4twCrg1C51jgSeDJbX9rBdREQGWCaBPhHYkrZeG5Sl+zNwWrD8v4DhZja664HM7CIzqzGzmrq6uv1pr4iI9CJbF0X/BTjOzJ4HjgO2Asmuldx9ubtXu3t1ZWVllk4tIiIA0QzqbAUmp61PCsraufs2gh66mQ0DTnf3+mw1UkRE+pdJD/05YKqZTTGzQuBM4MH0CmY2xszajnU1cEd2mykiIv3pN9DdPQFcAjwCvALc4+4vm9n1ZnZKUG0e8JqZvQ6MBb4/QO0VEZFemLvn5MTV1dVeU1OTk3OLiOQrM9vg7tU9bdM3RUVEQkKBLiISEgp0EZGQUKCLiISEAl1EJCQU6CIiIaFAFxEJCQW6iEhIKNBFREJCgS4iEhIKdBGRkFCgi4iEhAJdRCQkFOgiIiGhQBcRCQkFuohISCjQRURCQoEuIhISCnQRkZBQoIuIhIQCXUQkJBToIiIhoUAXEQkJBbqISEgo0EVEQkKBLiISEgp0EZGQUKCLiISEAl1EJCQU6CIiIaFAFxEJCQW6iEhIKNBFREIimusGiEhm4vE4tbW1NDU15bopMgiKi4uZNGkSsVgs430U6CJ5ora2luHDh1NVVYWZ5bo5MoDcnZ07d1JbW8uUKVMy3i+jIRczm29mr5nZZjNb0sP2Q81srZk9b2YvmNnJ+9B2EclAU1MTo0ePVpgPAWbG6NGj9/ndWL+BbmYR4CbgJOBIYLGZHdml2reBe9x9NnAmcPM+tUJEMqIwHzr252+dSQ/9GGCzu7/p7i3AKuDULnUcGBEsjwS27XNLRETkgGQS6BOBLWnrtUFZuqXA2WZWCzwMXNrTgczsIjOrMbOaurq6/WiuiORKfX09N9+872++Tz75ZOrr6/usc+211/L444/vb9MkkK2PLS4G7nT3ScDJwC/NrNux3X25u1e7e3VlZWWWTi0ig6G3QE8kEn3u9/DDD1NeXt5nneuvv56///u/P6D25Up/j38wZfIpl63A5LT1SUFZuq8C8wHcfb2ZFQNjgB3ZaKSIdPbd373Mpm27s3rMIyeM4LovTO91+5IlS/jLX/7CrFmziMViFBcXU1FRwauvvsrrr7/OF7/4RbZs2UJTUxOXX345F110EQBVVVXU1NTQ2NjISSedxGc/+1meeeYZJk6cyG9/+1tKSko477zzWLBgAV/60peoqqri3HPP5Xe/+x3xeJx7772XI444grq6Or785S+zbds2/vZv/5bHHnuMDRs2MGbMmB7b21t71qxZwzXXXEMymWTMmDE88cQTNDY2cumll1JTU4OZcd1113H66aczbNgwGhsbAbjvvvt46KGHuPPOOznvvPMoLi7m+eefZ+7cuZx55plcfvnlNDU1UVJSwooVK/jEJz5BMpnkqquuYs2aNRQUFHDhhRcyffp0brzxRh544AEAHnvsMW6++WZWr159wH/DTAL9OWCqmU0hFeRnAl/uUucd4ATgTjObBhQDGlMRCZEf/OAHvPTSS2zcuJGnnnqKz3/+87z00kvtH6u74447GDVqFHv37uVTn/oUp59+OqNHj+50jDfeeINf//rX3HbbbZxxxhncf//9nH322d3ONWbMGP70pz9x8803c8MNN3D77bfz3e9+l+OPP56rr76aNWvW8POf/7zP9vbUntbWVi688ELWrVvHlClT+OCDDwD43ve+x8iRI3nxxRcB2LVrV7/PR21tLc888wyRSITdu3fz+9//nmg0yuOPP84111zD/fffz/Lly3nrrbfYuHEj0WiUDz74gIqKCv7pn/6Juro6KisrWbFiBV/5ylcy+hv0p99Ad/eEmV0CPAJEgDvc/WUzux6ocfcHgX8GbjOzK0hdID3P3T0rLRSRbvrqSQ+WY445ptNnpG+88cb2XuaWLVt44403ugX6lClTmDVrFgBz5szhrbfe6vHYp512Wnud3/zmNwA8/fTT7cefP38+FRUVfbavp/bU1dXxuc99rr3do0aNAuDxxx9n1apV7fv2d2yAhQsXEolEAGhoaODcc8/ljTfewMyIx+Ptx7344ouJRqOdznfOOefwq1/9ivPPP5/169ezcuXKfs+XiYy+WOTuD5O62Jledm3a8iZgblZaJCJ5oaysrH35qaee4vHHH2f9+vWUlpYyb968Hj9DXVRU1L4ciUTYu3dvj8duqxeJRPZrjDrT9vQn/aODXfdPf/zf+c53+Lu/+ztWr17NW2+9xbx58/o87vnnn88XvvAFiouLWbhwYXvgHyjN5SIiGRk+fDgffvhhj9saGhqoqKigtLSUV199lWeffTbr5587dy733HMPAI8++mifwyK9tefYY49l3bp1/PWvfwVoH3I58cQTuemmm9r3bzv22LFjeeWVV2htbe1zjLuhoYGJE1Mf/rvzzjvby0888URuvfXW9heltvNNmDCBCRMmsGzZMs4///x9eh76okAXkYyMHj2auXPnctRRR/HNb36z07b58+eTSCSYNm0aS5Ys4dhjj836+a+77joeffRRjjrqKO69917GjRvH8OHDe6zbW3sqKytZvnw5p512GjNnzmTRokUAfPvb32bXrl0cddRRzJw5k7Vr1wKp6wYLFizgM5/5DOPHj++1bd/61re4+uqrmT17dqd3FBdccAGHHnooRx99NDNnzuTuu+9u33bWWWcxefJkpk2bdsDPTRvL1VB3dXW119TU5OTcIvnolVdeyeo/f75pbm4mEokQjUZZv349X/va19i4cWOum7XfLrnkEmbPns1Xv/rVXuv09Dc3sw3uXt1TfU3OJSJ54Z133uGMM86gtbWVwsJCbrvttlw3ab/NmTOHsrIyfvzjH2f1uAp0EckLU6dO5fnnn+9UtnPnTk444YRudZ944olun7A5mGzYsGFAjqtAF5G8NXr06Lwedsk2XRQVEQkJBbqISEgo0EVEQkKBLiISEgp0ERkQw4YNy3UThhwFuoiE2sE0X/lA08cWRfLRfy6Bd1/M7jHHzYCTftDr5iVLljB58mS+/vWvA7B06VKi0Shr165l165dxONxli1bxqmndv2Fyu4aGxs59dRTe9xv5cqV3HDDDZgZRx99NL/85S957733uPjii3nzzTcBuOWWW5gwYQILFizgpZdeAuCGG26gsbGRpUuXMm/ePGbNmsXTTz/N4sWLOfzww1m2bBktLS2MHj2au+66i7Fjx/Y4D3pDQwMvvPACP/3pTwG47bbb2LRpEz/5yU8O6OkdDAp0EcnIokWL+MY3vtEe6Pfccw+PPPIIl112GSNGjOD999/n2GOP5ZRTTun3B46Li4tZvXp1t/02bdrEsmXLeOaZZxgzZkz7ZFaXXXYZxx13HKtXryaZTNLY2NjvnOUtLS20TS+ya9cunn32WcyM22+/nR/+8If8+Mc/7nEe9Fgsxve//31+9KMfEYvFWLFiBbfeeuuBPn2DQoEuko/66EkPlNmzZ7Njxw62bdtGXV0dFRUVjBs3jiuuuIJ169ZRUFDA1q1bee+99xg3blyfx3J3rrnmmm77PfnkkyxcuLD9V4ja5g9/8skn2+cMj0QijBw5st9Ab5t4C1I/RrFo0SK2b99OS0tL+3zovc2Dfvzxx/PQQw8xbdo04vE4M2bM2MdnKzcU6CKSsYULF3Lffffx7rvvsmjRIu666y7q6urYsGEDsViMqqqqjOYd39/90kWjUVpbW9vX+5qv/NJLL+XKK6/klFNO4amnnmLp0qV9HvuCCy7g3/7t3zjiiCOyOr3tQNNFURHJ2KJFi1i1ahX33XcfCxcupKGhgUMOOYRYLMbatWt5++23MzpOb/sdf/zx3HvvvezcuRPomD/8hBNO4JZbbgEgmUzS0NDA2LFj2bFjBzt37qS5uZmHHnqoz/O1zVf+i1/8or28t3nQP/3pT7NlyxbuvvtuFi9enOnTk3MKdBHJ2PTp0/nwww+ZOHEi48eP56yzzqKmpoYZM2awcuVKjjjiiIyO09t+06dP51//9V857rjjmDlzJldeeSUAP/vZz1i7di0zZsxgzpw5bNq0iVgsxrXXXssxxxzDiSee2Oe5ly5dysKFC5kzZ06nH5XubR50gDPOOIO5c+dm9HN0BwvNhy6SJ4b6fOiDbcGCBVxxxRU9zuY4WPZ1PnT10EVE0tTX13P44YdTUlKS0zDfH7ooKiID5sUXX+Scc87pVFZUVMQf/vCHHLWof+Xl5bz++uu5bsZ+UaCL5BF37/cz3geTGTNmaL7y/bQ/w+EachHJE8XFxezcuXO//tElv7g7O3fupLi4eJ/2Uw9dJE9MmjSJ2tpa6urqct0UGQTFxcVMmjRpn/ZRoIvkiVgs1v4NR5GeaMhFRCQkFOgiIiGhQBcRCQkFuohISCjQRURCQoEuIhISCnQRkZBQoIuIhIQCXUQkJBToIiIhkVGgm9l8M3vNzDab2ZIetv/EzDYGt9fNrD77TRURkb70O5eLmUWAm4ATgVrgOTN70N03tdVx9yvS6l8KzB6AtoqISB8y6aEfA2x29zfdvQVYBZzaR/3FwK+z0TgREclcJoE+EdiStl4blHVjZocBU4Ane9l+kZnVmFmNpgAVEcmubF8UPRO4z92TPW109+XuXu3u1ZWVlVk+tYjI0JZJoG8FJqetTwrKenImGm4REcmJTAL9OWCqmU0xs0JSof1g10pmdgRQAazPbhNFRCQT/Qa6uyeAS4BHgFeAe9z9ZTO73sxOSat6JrDK9YOHIiI5kdFP0Ln7w8DDXcqu7bK+NHvNEhGRfaVvioqIhIQCXUQkJBToIiIhoUAXEQkJBbqISEgo0EVEQkKBLiISEgp0EZGQUKCLiISEAl1EJCQU6CIiIaFAFxEJCQW6iEhIKNBFREJCgS4iEhIKdBGRkFCgi4iEhAJdRCQkFOgiIiGhQBcRCQkFuohISCjQRURCQoEuIhISCnQRkZBQoIuIhIQCXUQkJBToIiIhoUAXEQkJBbqISEgo0EVEQkKBLiISEgp0EZGQUKCLiISEAl1EJCQU6CIiIZFRoJvZfDN7zcw2m9mSXuqcYWabzOxlM7s7u80UEZH+RPurYGYR4CbgRKAWeM7MHnT3TWl1pgJXA3PdfZeZHTJQDRYRkZ5l0kM/Btjs7m+6ewuwCji1S50LgZvcfReAu+/IbjNFRKQ/mQT6RGBL2nptUJbucOBwM/tvM3vWzOb3dCAzu8jMasyspq6ubv9aLCIiPcrWRdEoMBWYBywGbjOz8q6V3H25u1e7e3VlZWWWTi0iIpBZoG8FJqetTwrK0tUCD7p73N3/CrxOKuBFRGSQZBLozwFTzWyKmRUCZwIPdqnzAKneOWY2htQQzJtZbKeIiPSj30B39wRwCfAI8Apwj7u/bGbXm9kpQbVHgJ1mtglYC3zT3XcOVKNFRKQ7c/ecnLi6utprampycm4RkXxlZhvcvbqnbfqmqIhISCjQRURCQoEuIhISCnQRkZBQoIuIhIQCXUQkJBToIiIhoUAXEQkJBbqISEgo0EVEQkKBLiISEgp0EZGQUKCLiIREvz8SLYPMHZJxSDZDogUSTalld4jEIFLY5RYDs1y3WkQOAgp06D1E25dbINEclAW3ZLAt0RKUpy93rdNT/fQ6zZ2Pzz5Oadwt5Ashmhb4kaKO5WhR2gtD2nKn8p6Osy/H71JekMM3gq3J1HOcjENrIm05HvzN46my1kQGy3FIJtKW492PlfFyS5djBctmUBCDSDR1XxBNPZft9wO8ra08421djlcQVQcjh/Iv0Lf/Gd75wz6EaHpYdgnoAwnR3kSLU0EWLQyWCztCLlqcKi8aniqLFgV1i7rXiRR1XraCjiBItASBkHbrqaxTeRxaGmHvB6nlRHNHUCXTl1uy8zyks0jaC8Y+vLhgGYRtT2Gdtpytv2uvj60geNEKgq+/5WghRIZ1hGF6MEZiqc5FW8C3JtKW4x2PLdECrXt63tZ233XbQD8P6QqivYd9ry8SkY46Xdct0n+dTuuRDOqkHzsbxz04XsjyL9Df/C947Dudy7qGX08B2S1EuwRut3BNr5Nh/TAMf7S/W9nXF4z0W7zjBbXtnU/XF5dkc/e6iWZo3t1xfm8NgjDa8SJQEIVYaVpYFPYQjG37ZLJcSLdQzWS5LaBz+e5jX7QmO94dtCZ6eSHoaVvaC0Nf25Lx1Dn2Z5u3pu7je4MXo0RQP5F2a+2ynr49nutnN8UKMn+hmHcVHHV61puQf4Fe/RWY9eWOsA5DiB5MzIIXr8Jct0SyqSAIGIpz3ZKB0Rb4nuzjRSHZQ3kvdTI6Tm/H7LLuPdQpqRiQpyH/Ar1oWOomItKmoAAK1AnJk/eLIiLSHwW6iEhIKNBFREJCgS4iEhIKdBGRkFCgi4iEhAJdRCQkFOgiIiFh7oM4x0P6ic3qgLf3c/cxwPtZbE4+0GMeGvSYh4YDecyHuXtlTxtyFugHwsxq3L061+0YTHrMQ4Me89AwUI9ZQy4iIiGhQBcRCYl8DfTluW5ADugxDw16zEPDgDzmvBxDFxGR7vK1hy4iIl0o0EVEQiLvAt3M5pvZa2a22cyW5Lo9A83M7jCzHWb2Uq7bMljMbLKZrTWzTWb2spldnus2DTQzKzazP5rZn4PH/N1ct2kwmFnEzJ43s4dy3ZbBYGZvmdmLZrbRzGqyfvx8GkM3swjwOnAiUAs8Byx29005bdgAMrPPAY3ASnc/KtftGQxmNh4Y7+5/MrPhwAbgiyH/OxtQ5u6NZhYDngYud/dnc9y0AWVmVwLVwAh3X5Dr9gw0M3sLqHb3AfkiVb710I8BNrv7m+7eAqwCTs1xmwaUu68DPsh1OwaTu2939z8Fyx8CrwATc9uqgeUpjcFqLLjlT29rP5jZJODzwO25bktY5FugTwS2pK3XEvJ/9KHOzKqA2cAfctuSgRcMP2wEdgCPuXvYH/NPgW8BrbluyCBy4FEz22BmF2X74PkW6DKEmNkw4H7gG+6+O9ftGWjunnT3WcAk4BgzC+0Qm5ktAHa4+4Zct2WQfdbdPwmcBHw9GFLNmnwL9K3A5LT1SUGZhEwwjnw/cJe7/ybX7RlM7l4PrAXm57otA2gucEowprwKON7MfpXbJg08d98a3O8AVpMaRs6afAv054CpZjbFzAqBM4EHc9wmybLgAuHPgVfc/X/nuj2Dwcwqzaw8WC4hdeH/1dy2auC4+9XuPsndq0j9Hz/p7mfnuFkDyszKgov8mFkZ8D+BrH56La8C3d0TwCXAI6QulN3j7i/ntlUDy8x+DawHPmFmtWb21Vy3aRDMBc4h1WvbGNxOznWjBth4YBWDK5QAAAIFSURBVK2ZvUCq4/KYuw+Jj/INIWOBp83sz8Afgf/n7muyeYK8+tiiiIj0Lq966CIi0jsFuohISCjQRURCQoEuIhISCnQRkZBQoEvomFky7eOOG7M5K6eZVQ2lmS8lv0Rz3QCRAbA3+Aq9yJCiHroMGcFc1D8M5qP+o5l9PCivMrMnzewFM3vCzA4Nysea2epgjvI/m9lngkNFzOy2YN7yR4NvdmJmlwVzuL9gZqty9DBlCFOgSxiVdBlyWZS2rcHdZwD/QWq2P4D/A/zC3Y8G7gJuDMpvBP7L3WcCnwTavpU8FbjJ3acD9cDpQfkSYHZwnIsH6sGJ9EbfFJXQMbNGdx/WQ/lbwPHu/mYw+de77j7azN4n9YMa8aB8u7uPMbM6YJK7N6cdo4rU1/KnButXATF3X2Zma0j9GMkDwANp85uLDAr10GWo8V6W90Vz2nKSjmtRnwduItWbf87MdI1KBpUCXYaaRWn364PlZ0jN+AdwFvD7YPkJ4GvQ/uMTI3s7qJkVAJPdfS1wFTAS6PYuQWQgqQchYVQS/PJPmzXu3vbRxYpgRsNmYHFQdimwwsy+CdQB5wfllwPLgxkuk6TCfXsv54wAvwpC34Abg3nNRQaNxtBlyBjoH+gVyTUNuYiIhIR66CIiIaEeuohISCjQRURCQoEuIhISCnQRkZBQoIuIhMT/B3x9d9QFD/6lAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_bthcofgHvEk"
      },
      "source": [
        "Looking at the loss curves we can say that the **feature extractor model** which was trained for 10 epochs performing better than the **fine tune model**. \n",
        "\n",
        "Though the results might get worse if we fine tune the feature extractor model."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RvT60JXuSBmF"
      },
      "source": [
        "### 2. Recreate the first model (the feature extraction model) with mixed_precision turned on."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7KaeKDpwSbAR",
        "outputId": "c8957b99-73e0-4cef-ff44-51b840f4bc07",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "# Set global policy to mixed precision\n",
        "from tensorflow.keras import mixed_precision\n",
        "mixed_precision.set_global_policy(policy=\"mixed_float16\") "
      ],
      "execution_count": 24,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "INFO:tensorflow:Mixed precision compatibility check (mixed_float16): OK\n",
            "Your GPU will likely run quickly with dtype policy mixed_float16 as it has compute capability of at least 7.0. Your GPU: Tesla T4, compute capability 7.5\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "uo7ZKlshUoSU"
      },
      "source": [
        "class_names = train_data_all_10_percent.class_names"
      ],
      "execution_count": 25,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "j0lOUVrsenwS"
      },
      "source": [
        "# Building a model with mixed precision \n",
        "\n",
        "# Create base model\n",
        "input_shape = (224, 224, 3)\n",
        "base_model_fp = tf.keras.applications.EfficientNetB0(include_top=False)\n",
        "base_model_fp.trainable = False \n",
        "\n",
        "# Create Functional model \n",
        "inputs = layers.Input(shape=input_shape, name=\"input_layer\")\n",
        "\n",
        "x = base_model_fp(inputs, training=False) # set base_model to inference mode only\n",
        "x = layers.GlobalAveragePooling2D(name=\"pooling_layer\")(x)\n",
        "x = layers.Dense(len(class_names))(x) \n",
        "\n",
        "# Separate activation of output layer so we can output float32 activations\n",
        "outputs = layers.Activation(\"softmax\", dtype=tf.float32, name=\"softmax_float32\")(x) \n",
        "mixed_pre_model = tf.keras.Model(inputs, outputs)\n",
        "\n",
        "# Compile the model\n",
        "mixed_pre_model.compile(loss=\"categorical_crossentropy\",\n",
        "              optimizer=tf.keras.optimizers.Adam(),\n",
        "              metrics=[\"accuracy\"])\n"
      ],
      "execution_count": 26,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "X4el7IWIj18q",
        "outputId": "a01156d3-c1d9-4817-ed1f-bcb87dbcb95f"
      },
      "source": [
        "# Summary of the model \n",
        "mixed_pre_model.summary()\n"
      ],
      "execution_count": 27,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Model: \"model\"\n",
            "_________________________________________________________________\n",
            "Layer (type)                 Output Shape              Param #   \n",
            "=================================================================\n",
            "input_layer (InputLayer)     [(None, 224, 224, 3)]     0         \n",
            "_________________________________________________________________\n",
            "efficientnetb0 (Functional)  (None, None, None, 1280)  4049571   \n",
            "_________________________________________________________________\n",
            "pooling_layer (GlobalAverage (None, 1280)              0         \n",
            "_________________________________________________________________\n",
            "dense (Dense)                (None, 101)               129381    \n",
            "_________________________________________________________________\n",
            "softmax_float32 (Activation) (None, 101)               0         \n",
            "=================================================================\n",
            "Total params: 4,178,952\n",
            "Trainable params: 129,381\n",
            "Non-trainable params: 4,049,571\n",
            "_________________________________________________________________\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "NH51WrSrkuZu",
        "outputId": "7e951bab-b000-4628-91e2-6aac883b6c16"
      },
      "source": [
        "initial_epochs\n"
      ],
      "execution_count": 28,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "5"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 28
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "7u8zAAVYj_la",
        "outputId": "3207e9f8-3df5-4ca4-fc5b-edff868fd204"
      },
      "source": [
        "# Fitting the feature extractor model for 5 epochs                                          \n",
        "history_mixed_feature_extractor = mixed_pre_model.fit(train_data_all_10_percent , \n",
        "                                                      epochs= initial_epochs, \n",
        "                                                      validation_data=test_data,\n",
        "                                                      validation_steps=int(0.15 * len(test_data)))"
      ],
      "execution_count": 29,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Epoch 1/5\n",
            "237/237 [==============================] - 51s 190ms/step - loss: 3.7659 - accuracy: 0.1993 - val_loss: 2.4225 - val_accuracy: 0.4656\n",
            "Epoch 2/5\n",
            "237/237 [==============================] - 44s 184ms/step - loss: 1.9215 - accuracy: 0.5805 - val_loss: 2.0560 - val_accuracy: 0.5019\n",
            "Epoch 3/5\n",
            "237/237 [==============================] - 44s 183ms/step - loss: 1.4544 - accuracy: 0.6755 - val_loss: 1.9009 - val_accuracy: 0.5201\n",
            "Epoch 4/5\n",
            "237/237 [==============================] - 44s 185ms/step - loss: 1.2018 - accuracy: 0.7248 - val_loss: 1.8285 - val_accuracy: 0.5312\n",
            "Epoch 5/5\n",
            "237/237 [==============================] - 44s 184ms/step - loss: 1.0222 - accuracy: 0.7755 - val_loss: 1.7676 - val_accuracy: 0.5410\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "PUKM6E6YkzoZ"
      },
      "source": [
        "# Unfreezing the layers for fine tuning the model \n",
        "base_model_fp.trainable = True \n",
        "\n",
        "# Refreeze every layer except for the last 7 \n",
        "for layer in base_model_fp.layers[:-5]:\n",
        "  layer.trainable = False"
      ],
      "execution_count": 30,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ocSJXxwVnAxQ"
      },
      "source": [
        "# Re-compiling the model \n",
        "mixed_pre_model.compile(loss = tf.keras.losses.CategoricalCrossentropy() , \n",
        "                        optimizer = tf.keras.optimizers.Adam(learning_rate= 0.0001) , \n",
        "                        metrics = ['accuracy'])"
      ],
      "execution_count": 31,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ENdoiFkBnPIP",
        "outputId": "20b9e438-d7c8-4368-8d8c-f9fdec00c1d1"
      },
      "source": [
        "# Check which layers are trainable\n",
        "for layer_number, layer in enumerate(base_model_fp.layers):\n",
        "  print(layer_number, layer.name, layer.trainable)"
      ],
      "execution_count": 32,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "0 input_3 False\n",
            "1 rescaling_2 False\n",
            "2 normalization_2 False\n",
            "3 stem_conv_pad False\n",
            "4 stem_conv False\n",
            "5 stem_bn False\n",
            "6 stem_activation False\n",
            "7 block1a_dwconv False\n",
            "8 block1a_bn False\n",
            "9 block1a_activation False\n",
            "10 block1a_se_squeeze False\n",
            "11 block1a_se_reshape False\n",
            "12 block1a_se_reduce False\n",
            "13 block1a_se_expand False\n",
            "14 block1a_se_excite False\n",
            "15 block1a_project_conv False\n",
            "16 block1a_project_bn False\n",
            "17 block2a_expand_conv False\n",
            "18 block2a_expand_bn False\n",
            "19 block2a_expand_activation False\n",
            "20 block2a_dwconv_pad False\n",
            "21 block2a_dwconv False\n",
            "22 block2a_bn False\n",
            "23 block2a_activation False\n",
            "24 block2a_se_squeeze False\n",
            "25 block2a_se_reshape False\n",
            "26 block2a_se_reduce False\n",
            "27 block2a_se_expand False\n",
            "28 block2a_se_excite False\n",
            "29 block2a_project_conv False\n",
            "30 block2a_project_bn False\n",
            "31 block2b_expand_conv False\n",
            "32 block2b_expand_bn False\n",
            "33 block2b_expand_activation False\n",
            "34 block2b_dwconv False\n",
            "35 block2b_bn False\n",
            "36 block2b_activation False\n",
            "37 block2b_se_squeeze False\n",
            "38 block2b_se_reshape False\n",
            "39 block2b_se_reduce False\n",
            "40 block2b_se_expand False\n",
            "41 block2b_se_excite False\n",
            "42 block2b_project_conv False\n",
            "43 block2b_project_bn False\n",
            "44 block2b_drop False\n",
            "45 block2b_add False\n",
            "46 block3a_expand_conv False\n",
            "47 block3a_expand_bn False\n",
            "48 block3a_expand_activation False\n",
            "49 block3a_dwconv_pad False\n",
            "50 block3a_dwconv False\n",
            "51 block3a_bn False\n",
            "52 block3a_activation False\n",
            "53 block3a_se_squeeze False\n",
            "54 block3a_se_reshape False\n",
            "55 block3a_se_reduce False\n",
            "56 block3a_se_expand False\n",
            "57 block3a_se_excite False\n",
            "58 block3a_project_conv False\n",
            "59 block3a_project_bn False\n",
            "60 block3b_expand_conv False\n",
            "61 block3b_expand_bn False\n",
            "62 block3b_expand_activation False\n",
            "63 block3b_dwconv False\n",
            "64 block3b_bn False\n",
            "65 block3b_activation False\n",
            "66 block3b_se_squeeze False\n",
            "67 block3b_se_reshape False\n",
            "68 block3b_se_reduce False\n",
            "69 block3b_se_expand False\n",
            "70 block3b_se_excite False\n",
            "71 block3b_project_conv False\n",
            "72 block3b_project_bn False\n",
            "73 block3b_drop False\n",
            "74 block3b_add False\n",
            "75 block4a_expand_conv False\n",
            "76 block4a_expand_bn False\n",
            "77 block4a_expand_activation False\n",
            "78 block4a_dwconv_pad False\n",
            "79 block4a_dwconv False\n",
            "80 block4a_bn False\n",
            "81 block4a_activation False\n",
            "82 block4a_se_squeeze False\n",
            "83 block4a_se_reshape False\n",
            "84 block4a_se_reduce False\n",
            "85 block4a_se_expand False\n",
            "86 block4a_se_excite False\n",
            "87 block4a_project_conv False\n",
            "88 block4a_project_bn False\n",
            "89 block4b_expand_conv False\n",
            "90 block4b_expand_bn False\n",
            "91 block4b_expand_activation False\n",
            "92 block4b_dwconv False\n",
            "93 block4b_bn False\n",
            "94 block4b_activation False\n",
            "95 block4b_se_squeeze False\n",
            "96 block4b_se_reshape False\n",
            "97 block4b_se_reduce False\n",
            "98 block4b_se_expand False\n",
            "99 block4b_se_excite False\n",
            "100 block4b_project_conv False\n",
            "101 block4b_project_bn False\n",
            "102 block4b_drop False\n",
            "103 block4b_add False\n",
            "104 block4c_expand_conv False\n",
            "105 block4c_expand_bn False\n",
            "106 block4c_expand_activation False\n",
            "107 block4c_dwconv False\n",
            "108 block4c_bn False\n",
            "109 block4c_activation False\n",
            "110 block4c_se_squeeze False\n",
            "111 block4c_se_reshape False\n",
            "112 block4c_se_reduce False\n",
            "113 block4c_se_expand False\n",
            "114 block4c_se_excite False\n",
            "115 block4c_project_conv False\n",
            "116 block4c_project_bn False\n",
            "117 block4c_drop False\n",
            "118 block4c_add False\n",
            "119 block5a_expand_conv False\n",
            "120 block5a_expand_bn False\n",
            "121 block5a_expand_activation False\n",
            "122 block5a_dwconv False\n",
            "123 block5a_bn False\n",
            "124 block5a_activation False\n",
            "125 block5a_se_squeeze False\n",
            "126 block5a_se_reshape False\n",
            "127 block5a_se_reduce False\n",
            "128 block5a_se_expand False\n",
            "129 block5a_se_excite False\n",
            "130 block5a_project_conv False\n",
            "131 block5a_project_bn False\n",
            "132 block5b_expand_conv False\n",
            "133 block5b_expand_bn False\n",
            "134 block5b_expand_activation False\n",
            "135 block5b_dwconv False\n",
            "136 block5b_bn False\n",
            "137 block5b_activation False\n",
            "138 block5b_se_squeeze False\n",
            "139 block5b_se_reshape False\n",
            "140 block5b_se_reduce False\n",
            "141 block5b_se_expand False\n",
            "142 block5b_se_excite False\n",
            "143 block5b_project_conv False\n",
            "144 block5b_project_bn False\n",
            "145 block5b_drop False\n",
            "146 block5b_add False\n",
            "147 block5c_expand_conv False\n",
            "148 block5c_expand_bn False\n",
            "149 block5c_expand_activation False\n",
            "150 block5c_dwconv False\n",
            "151 block5c_bn False\n",
            "152 block5c_activation False\n",
            "153 block5c_se_squeeze False\n",
            "154 block5c_se_reshape False\n",
            "155 block5c_se_reduce False\n",
            "156 block5c_se_expand False\n",
            "157 block5c_se_excite False\n",
            "158 block5c_project_conv False\n",
            "159 block5c_project_bn False\n",
            "160 block5c_drop False\n",
            "161 block5c_add False\n",
            "162 block6a_expand_conv False\n",
            "163 block6a_expand_bn False\n",
            "164 block6a_expand_activation False\n",
            "165 block6a_dwconv_pad False\n",
            "166 block6a_dwconv False\n",
            "167 block6a_bn False\n",
            "168 block6a_activation False\n",
            "169 block6a_se_squeeze False\n",
            "170 block6a_se_reshape False\n",
            "171 block6a_se_reduce False\n",
            "172 block6a_se_expand False\n",
            "173 block6a_se_excite False\n",
            "174 block6a_project_conv False\n",
            "175 block6a_project_bn False\n",
            "176 block6b_expand_conv False\n",
            "177 block6b_expand_bn False\n",
            "178 block6b_expand_activation False\n",
            "179 block6b_dwconv False\n",
            "180 block6b_bn False\n",
            "181 block6b_activation False\n",
            "182 block6b_se_squeeze False\n",
            "183 block6b_se_reshape False\n",
            "184 block6b_se_reduce False\n",
            "185 block6b_se_expand False\n",
            "186 block6b_se_excite False\n",
            "187 block6b_project_conv False\n",
            "188 block6b_project_bn False\n",
            "189 block6b_drop False\n",
            "190 block6b_add False\n",
            "191 block6c_expand_conv False\n",
            "192 block6c_expand_bn False\n",
            "193 block6c_expand_activation False\n",
            "194 block6c_dwconv False\n",
            "195 block6c_bn False\n",
            "196 block6c_activation False\n",
            "197 block6c_se_squeeze False\n",
            "198 block6c_se_reshape False\n",
            "199 block6c_se_reduce False\n",
            "200 block6c_se_expand False\n",
            "201 block6c_se_excite False\n",
            "202 block6c_project_conv False\n",
            "203 block6c_project_bn False\n",
            "204 block6c_drop False\n",
            "205 block6c_add False\n",
            "206 block6d_expand_conv False\n",
            "207 block6d_expand_bn False\n",
            "208 block6d_expand_activation False\n",
            "209 block6d_dwconv False\n",
            "210 block6d_bn False\n",
            "211 block6d_activation False\n",
            "212 block6d_se_squeeze False\n",
            "213 block6d_se_reshape False\n",
            "214 block6d_se_reduce False\n",
            "215 block6d_se_expand False\n",
            "216 block6d_se_excite False\n",
            "217 block6d_project_conv False\n",
            "218 block6d_project_bn False\n",
            "219 block6d_drop False\n",
            "220 block6d_add False\n",
            "221 block7a_expand_conv False\n",
            "222 block7a_expand_bn False\n",
            "223 block7a_expand_activation False\n",
            "224 block7a_dwconv False\n",
            "225 block7a_bn False\n",
            "226 block7a_activation False\n",
            "227 block7a_se_squeeze False\n",
            "228 block7a_se_reshape False\n",
            "229 block7a_se_reduce False\n",
            "230 block7a_se_expand False\n",
            "231 block7a_se_excite False\n",
            "232 block7a_project_conv True\n",
            "233 block7a_project_bn True\n",
            "234 top_conv True\n",
            "235 top_bn True\n",
            "236 top_activation True\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "HlXgVxyHncuV",
        "outputId": "82124d3a-f37b-4702-823f-2ca3c66b175a"
      },
      "source": [
        "# Fine tuning the model for another 5 epochs \n",
        "fine_tune_epochs = initial_epochs + 5                                    \n",
        "history_mixed_model_fine_tune = mixed_pre_model.fit(train_data_all_10_percent,\n",
        "                                                     epochs=fine_tune_epochs,\n",
        "                                                     validation_data=test_data,\n",
        "                                                     validation_steps=int(0.15 * len(test_data)), \n",
        "                                                     initial_epoch=history_mixed_feature_extractor.epoch[-1]) "
      ],
      "execution_count": 33,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Epoch 5/10\n",
            "237/237 [==============================] - 51s 191ms/step - loss: 0.7952 - accuracy: 0.8197 - val_loss: 1.7345 - val_accuracy: 0.5530\n",
            "Epoch 6/10\n",
            "237/237 [==============================] - 44s 184ms/step - loss: 0.6214 - accuracy: 0.8539 - val_loss: 1.7504 - val_accuracy: 0.5543\n",
            "Epoch 7/10\n",
            "237/237 [==============================] - 44s 185ms/step - loss: 0.5337 - accuracy: 0.8797 - val_loss: 1.7703 - val_accuracy: 0.5564\n",
            "Epoch 8/10\n",
            "237/237 [==============================] - 44s 184ms/step - loss: 0.4586 - accuracy: 0.9005 - val_loss: 1.8039 - val_accuracy: 0.5561\n",
            "Epoch 9/10\n",
            "237/237 [==============================] - 44s 185ms/step - loss: 0.3898 - accuracy: 0.9205 - val_loss: 1.8105 - val_accuracy: 0.5561\n",
            "Epoch 10/10\n",
            "237/237 [==============================] - 44s 185ms/step - loss: 0.3393 - accuracy: 0.9332 - val_loss: 1.8379 - val_accuracy: 0.5543\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2TbmqX0lnz7F"
      },
      "source": [
        "- **Does it make the model train faster?**\n",
        "    \n",
        "  Not much, but it saved us some seconds. \n",
        "\n",
        "- **Does it effect the accuracy or performance of our model?**\n",
        "\n",
        "  By 1% not that much tho. \n",
        "- **What's the advatanges of using mixed_precision training?**\n",
        "\n",
        "  The advantages of mixed precision is evident when we are training pretty big models for longer epochs. In that case, we can spot a huge difference in our training time. \n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hiu4Fl9XYhw4"
      },
      "source": [
        "### 3. Take 3 of your own photos of food and use the trained model to make predictions on them, share your predictions with the other students in Discord and show off your Food Vision model 🍔👁.\n",
        "\n",
        "Now its upto you guys to make predictions with your custom picture and share the results on Discord. Good Luck. \n",
        "\n",
        "> **Note:** This is just a simple template for the solution and there is always room for improvements. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "cxDzombfaJMg"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    }
  ]
}